Error in tBigQueryOutput component

[Anonymous]

Hi All,

I have developed a talend job having only three components for insert data into bigquery tables using a csv file. This job is running and inserting data into bigquery table but having below error.

Starting job testjob_002 at 13:29 02/11/2018.

[WARN ]: org.jets3t.service.utils.Mimetypes - Unable to find 'mime.types' file in classpath
Starting load job.
[FATAL]: tal_cloud.testjob_002_0_1.testjob_002 - tBigQueryOutput_1_tBQBE 404 Not Found
{
"code" : 404,
"errors" : [ {
"domain" : "global",
"message" : "Not found: Job mytalendproject-220908:job_4pozTkcPzzMgeoFa4PnHmUz9fK2F",
"reason" : "notFound"
} ],
"message" : "Not found: Job mytalendproject-220908:job_4pozTkcPzzMgeoFa4PnHmUz9fK2F"

mytalendproject-220908:asia-northeast1.job_4pozTkcPzzMgeoFa4PnHmUz9fK2F
}
com.google.api.client.googleapis.json.GoogleJsonResponseException: 404 Not Found
{
"code" : 404,
"errors" : [ {
"domain" : "global",
"message" : "Not found: Job mytalendproject-220908:job_4pozTkcPzzMgeoFa4PnHmUz9fK2F",
"reason" : "notFound"
} ],
"message" : "Not found: Job mytalendproject-220908:job_4pozTkcPzzMgeoFa4PnHmUz9fK2F"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:145)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:312)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1049)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:410)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:343)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:460)
at tal_cloud.testjob_002_0_1.testjob_002.tFileInputDelimited_1Process(testjob_002.java:1474)
at tal_cloud.testjob_002_0_1.testjob_002.runJobInTOS(testjob_002.java:1845)
at tal_cloud.testjob_002_0_1.testjob_002.main(testjob_002.java:1695)
Exception in component tBigQueryOutput_1_tBQBE (testjob_002)
com.google.api.client.googleapis.json.GoogleJsonResponseException: 404 Not Found
{
"code" : 404,
"errors" : [ {
"domain" : "global",
"message" : "Not found: Job mytalendproject-220908:job_4pozTkcPzzMgeoFa4PnHmUz9fK2F",
"reason" : "notFound"
} ],
"message" : "Not found: Job mytalendproject-220908:job_4pozTkcPzzMgeoFa4PnHmUz9fK2F"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:145)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:312)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1049)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:410)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:343)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:460)
at tal_cloud.testjob_002_0_1.testjob_002.tFileInputDelimited_1Process(testjob_002.java:1474)
at tal_cloud.testjob_002_0_1.testjob_002.runJobInTOS(testjob_002.java:1845)
at tal_cloud.testjob_002_0_1.testjob_002.main(testjob_002.java:1695)
Job testjob_002 ended at 13:29 02/11/2018. [exit code=1]

I did not find any solution to remove this error.Please help me.

Thanks 

Kailash Yadav 

[Anonymous]

Hello Kailash Yadav,

Are you running big data sets? Is it a spark job? Can you please clarify in which Talend version/edition you are?

Best regards

Sabrina

[Anonymous]

Hi Sabrina,

Thanks for your reply. It's not a spark job its a simple DI Job we are using Talend Big data platform 6.5 version.

Thanks

Kailash Yadav  

[Anonymous]

Hi Sabrina,

I am getting the similar error in all the bigquery components for example 

tgooglebigqueryoutput

tgooglebigquerybulkexec

throwing job name not found error

{
"code" : 404,
"errors" : [ {
"domain" : "global",
"message" : "Not found: Job mytalendproject-220908:job_XpZYr_vRWRVbQONX7gDEGgv8jj9t",
"reason" : "notFound"
} ],
"message" : "Not found: Job mytalendproject-220908:job_XpZYr_vRWRVbQONX7gDEGgv8jj9t"
}

but the job gets executed on the google cloud platform.
the problem here it seems that the job name: "Job mytalendproject-220908:job_XpZYr_vRWRVbQONX7gDEGgv8jj9t"
is incorrect as it does not contain the locale

on the google bigquery the load job that gets executed is : "mytalendproject-220908:asia-northeast1.job_XpZYr_vRWRVbQONX7gDEGgv8jj9t"

Please help me it very critical now and having this issue from last two weeks.

Thanks

Kailash Yadav

[Anonymous]

Hello I have the same problem. 

Seems like it happens when the GCP project is not located either in the US or Europe. Outside those 2 regions you need to add an extra "location" value in the request. For exemple my GCP project is in Asia-east2 so he's looking for a job named : "MyProject_id":asia-east2."Myjobnumber" but Talend is sending a request with only "MyProject_id":"Myjobnumber" as it seems like we can't pass the location value in the tBigQueryOutput component. 

I'm still searching for an answer. Anyone can help ? 

Thanks

[Anonymous]

Hi,

    If you are using a subscription version of Talend, could you please request support team to create a JIRA ticket to verify this product feature? If you are using a free version of Talend, please use the below link to create the JIRA ticket.

https://jira.talendforge.org/secure/Dashboard.jspa

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved