topic Re: Google Dataproc Hive Connection throwing error in Talend Studio

Google Dataproc Hive Connection throwing error

ARajchandani — Sat, 16 Nov 2024 00:48:06 GMT

Hi Team,

Whenever I try to connect to Hive which is on Google Dataproc distribution, I get the following error:

com.google.api.client.googleapis.json.GoogleJsonResponseException: 404 Not Found

{

"code" : 404,

"errors" : [ {

"domain" : "global",

"message" : "Not found: Job projects/google-project-id/regions/global/jobs/mylocaltalendproject_W0ZfZr_0def4525-70cf-4355-96c3-2f5adcc2f77e_1",

"reason" : "notFound"

} ],

"message" : "Not found: Job

projects/google-project-id/regions/global/jobs/mylocaltalendproject

_W0ZfZr_0def4525-70cf-4355-96c3-2f5adcc2f77e_1",

"status" : "NOT_FOUND"

}

Re: Google Dataproc Hive Connection throwing error

Anonymous — Thu, 21 Jan 2021 04:18:50 GMT

Hello,

This normally means that the server received the request but could not find resource requested or that it doesn't exist.

What's Distributions Dataproc are you using?

Are you using Hive connection component and the option google credentials in a file? Screenshots of your connection setting will be preferred.

Best regards

Sabrina

Re: Google Dataproc Hive Connection throwing error

ARajchandani — Thu, 21 Jan 2021 06:31:34 GMT

Hi Sabrina, I have tried using tHiveConnection as well as the connection in tHiveInput as well but to no avail,

This is my flow

I am just trying to read some records from a table in Hive, tLibraryLoad is used to load the log4j library.

Here is the configuration for tHiveInput:

I can verify that the table and database name that I am giving exist in hive and even without providing the secret and access key, the j3 jar is getting transferred to the specified bucket.

Please let me know what I am doing wrong.

Re: Google Dataproc Hive Connection throwing error

ARajchandani — Fri, 22 Jan 2021 14:06:00 GMT

Also, I am using VM, for example "dataproc-m" name of my master dataproc node as the Cluster Identifier, am I doing this correct? Can really uses some help here

Re: Google Dataproc Hive Connection throwing error

Anonymous — Tue, 26 Jan 2021 07:56:39 GMT

Hello,

From your screenshots, you are using the option google credentials in a file.

Are you able to connect to and submit the queries using tHiverow? Could you please clarify in which Talend version/edition you are?

Best regards

Sabrina

Re: Google Dataproc Hive Connection throwing error

ARajchandani — Tue, 26 Jan 2021 18:54:06 GMT

tHiveRow is giving me the same error, I am using Talend Open Studio for Big Data (7.3.1)