tgooglebigqueryinput delete data

[Anonymous]

I want to delete data from google big query using delete statement on input element. I get error, but in google bigquery data is deleted. Can you explain why this error happens and how to overcome it? i am using Talend Open studio for data integration 7.2.1.20190131_1157-M2

Exception in component tBigQueryInput_2 (criteo_data_import)
com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "API limit exceeded: Unable to return a row that exceeds the API limits. To retrieve the row, export the table.",
"reason" : "apiLimitExceeded"
} ],
"message" : "API limit exceeded: Unable to return a row that exceeds the API limits. To retrieve the row, export the table.",
"status" : "INVALID_ARGUMENT"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:145)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:312)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1049)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:410)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:343)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:460)
at pigu_robertas.criteo_data_import_0_1.criteo_data_import.tBigQueryInput_2Process(criteo_data_import.java:12122)
at pigu_robertas.criteo_data_import_0_1.criteo_data_import.tFileInputDelimited_6Process(criteo_data_import.java:11680)
at pigu_robertas.criteo_data_import_0_1.criteo_data_import.tFileInputDelimited_5Process(criteo_data_import.java:10045)
at pigu_robertas.criteo_data_import_0_1.criteo_data_import.tFileInputDelimited_4Process(criteo_data_import.java:8410)
at pigu_robertas.criteo_data_import_0_1.criteo_data_import.tBigQueryInput_1Process(criteo_data_import.java:5893)
at pigu_robertas.criteo_data_import_0_1.criteo_data_import.tFileInputDelimited_3Process(criteo_data_import.java:5352)
at pigu_robertas.criteo_data_import_0_1.criteo_data_import.tFileInputDelimited_2Process(criteo_data_import.java:3939)
at pigu_robertas.criteo_data_import_0_1.criteo_data_import.tFileInputDelimited_1Process(criteo_data_import.java:2526)
at pigu_robertas.criteo_data_import_0_1.criteo_data_import.tJava_1Process(criteo_data_import.java:1145)
at pigu_robertas.criteo_data_import_0_1.criteo_data_import.runJobInTOS(criteo_data_import.java:28113)
at pigu_robertas.criteo_data_import_0_1.criteo_data_import.main(criteo_data_import.java:27942)

[Anonymous]

I wanted to try out Rest API, but i did not find in google documentation how to delete data from tabele. i only found how to delete entire table.

[Anonymous]

Try setting the result size instead of auto to Large(with allowLargeResults)

Thanks,

Prabuj

[Anonymous]

i tried that. then GBQ creates temp tables and tbigqueryinput element says not found.

Which version of talend do you use? Maybe my build is buggy.

[Anonymous]

Kindly check your quota and limit for big query.

https://cloud.google.com/bigquery/quotas
https://cloud.google.com/bigquery/docs/writing-results#large-results

Thanks,
Prabuj.

[Anonymous]

can you give an example?

[Anonymous]

As you can see on the error message:

"message" : "API limit exceeded: Unable to return a row that exceeds the API limits. To retrieve the row, export the table.",
"reason" : "apiLimitExceeded"

 Kindly check your quota and limit of your api !

Refer the link 

https://cloud.google.com/bigquery/quotas

https://cloud.google.com/bigquery/docs/writing-results#large-results

Thanks,

Prabuj.

/Don't forget to give kudos/

[Anonymous]

i give up. delete operation should return nothing... 

[Anonymous]

Don't give up. Try Fixing it up, even it's a solved issue tbigqueryinput in JIRA from talend for DML. Definitely it should work !!!
First check your API, Did you change the settings to allow large sets and find the limits of your api ? if limit is there ! Next use your query in query section.

Thanks,
Prabuj

[Anonymous]

So i tested another thing. I tried to drop table. so now i get:

{
"code" : 404,
"errors" : [ {
"domain" : "global",
"message" : "Not found: Table sonorous-mix-211210:marketing_platforms_data.criteo_transactions20190612",
"reason" : "notFound"
} ],
"message" : "Not found: Table sonorous-mix-211210:marketing_platforms_data.criteo_transactions20190612",
"status" : "NOT_FOUND"
}

It looks like after these operations component is calling second time.

[Anonymous]

Secondly: i transfered my data from partitioned table to simple table. no errors. So it also can be something with partitioned tables.