topic Re: Hive - Permissions needed to retrieve table schemas? in Talend Studio

Hive - Permissions needed to retrieve table schemas?

talendtester — Fri, 15 Nov 2024 22:07:12 GMT

I have successfully connected Talend to Hive, I can see the database, and I pulled data from a table in the database to a file via my Talend job.

When I attempt to retrieve schemas in the JDBC connection, I can click the database name but no tables are listed. It just fills the box next to the database name.

What permissions are needed in Hive for Talend to be able to see all the tables and retrieve schemas?

Apache Hive

3.1.3000.7.1.7.1000-141

HiveJDBC

02.06.13.1016

Re: Hive - Permissions needed to retrieve table schemas?

talendtester — Fri, 10 Feb 2023 21:18:42 GMT

@Shicong Hong @Xiaodi Shi Any ideas?

Re: Hive - Permissions needed to retrieve table schemas?

Anonymous — Tue, 14 Feb 2023 08:48:01 GMT

Hello,

It looks like this connection check has been done already and it is working from your side.

What is the error you are facing while retrieving schema? Could you please attach the screenshot and also the complete error log?

Is there seems to be latency when retrieving the schema?

Best regards

Sabrina

Re: Hive - Permissions needed to retrieve table schemas?

talendtester — Tue, 14 Feb 2023 16:13:23 GMT

The connection completes, there is no error.

There is not much to see, the tables names just don't show up under the database name.

I am pretty sure its a lack of permission, but don't know what Talend needs access to in order to request the permission.

I found this:

https://www.vertica.com/docs/9.3.x/HTML/Content/Authoring/HadoopIntegrationGuide/HCatalogConnector/ViewingHiveSchemaAndTableMetadata.htm

There are four system tables that contain metadata about the tables accessible through the HCatalog Connector:

HCATALOG_SCHEMATA lists all of the schemas that have been defined using the HCatalog Connector.
HCATALOG_TABLE_LIST contains an overview of all of the tables available from all schemas defined using the HCatalog Connector. This table only shows the tables that the user querying the table can access. The information in this table is retrieved using a single call to HiveServer2 for each schema defined using the HCatalog Connector, which means there is a little overhead when querying this table.
HCATALOG_TABLES contains more in-depth information than HCATALOG_TABLE_LIST.
HCATALOG_COLUMNS lists metadata about all of the columns in all of the tables available through the HCatalog Connector. As for HCATALOG_TABLES, querying this table results in one call to HiveServer2 per table, and therefore can take a while to complete.

Re: Hive - Permissions needed to retrieve table schemas?

Anonymous — Wed, 15 Feb 2023 04:46:36 GMT

Hello,

There will be some error messages in log if it is a permission issue.

Could you please open the error log generated automatically by the Studio to see whether you have error stack trace there; if you do, we will appreciate it a lot if you can paste it here.

About how to open this error log: in your Studio, click Window to open the menu, then select Show View->General->Error Log. Once you see an error, double click it.

Best regards

Sabrina