DELETE or INSERT query with tBigQueryInput component loads complete table content after command

Anonymous — Fri, 30 Aug 2019 08:51:15 GMT

Hello Talend-Community,

I've got a problem with the tBigQueryInput component. I am using talend to extract data from Google Analytics and push this data to Google Big Query every day. Within this workflow, I am deleting the entries from the last two days, in case of data changes and insert the fresh data.

Unfortunately, using the DELETE command via tBigQueryInput loads the whole raw table before the Component finishes. The problem here for me is, that some of my tables have tens of millios of rows - so this takes forever.

Does anyone know why this is the case and how to prevend this?

Thanks so much, looking forward to your feedback!

Re: DELETE or INSERT query with tBigQueryInput component loads complete table content after command

Anonymous — Mon, 02 Sep 2019 07:27:01 GMT

Hello,

Have you tried to specify the data that you want to delete or update? Would you mind posting your tBigQueryInput Delete statement here?

Starting from V 7.1.1, talend has already added support for "Standard SQL" in Bigquery.

Best regards

Sabrina

topic DELETE or INSERT query with tBigQueryInput component loads complete table content after command in Talend Studio

DELETE or INSERT query with tBigQueryInput component loads complete table content after command

Re: DELETE or INSERT query with tBigQueryInput component loads complete table content after command