topic Re: Talend Job Failure - Bigquery Inserts in Talend Studio

Talend Job Failure - Bigquery Inserts

techwh — Fri, 15 Nov 2024 23:54:28 GMT

Issue Summary:

Over the last couple of days when we try to insert data into Google Cloud Platform Bigquery using Talend the jobs are failing with the below error message,

.-------------------+------+--------+----------+-----------------+--------------------------------+-------+--------+--------------+-----------------+----------------------------------------------------------------------------------------------+----.

| tLogRow_Error |

|=------------------+------+--------+----------+-----------------+--------------------------------+-------+--------+--------------+-----------------+----------------------------------------------------------------------------------------------+---=|

|2021-07-29 06:56:12|4qhn1M|qN2Buf |qN2Buf ||Default|6 |Java Exception|tBigQuerySQLRow_2|com.google.cloud.bigquery.BigQueryException:Expected , or } after key:value pair.

l":fals

^|1 |

'-------------------+------+--------+----------+-----------------+--------------------------------+-------+--------+--------------+-----------------+----------------------------------------------------------------------------------------------+----'

Whenever this failure happens the data does not get loaded into the table and we had to do multiple reruns to load the data into the tables. Usually succeeds in consecutive attempts.

This error is happening at random not specific to any particular table. These insert statements were working very well for more than a year so I do not think there is some issue with the SQL statements.

We are just clueless if any setting got updated at Talend end recently.

Please find below the software version details,

JAVA Version :

java version "1.8.0_251"

Talend Development Platform :

Talend Cloud management Platform 7.2.1.20190909_1200_patch

The jobs are scheduled on Talend Management Console platform.

Re: Talend Job Failure - Bigquery Inserts

Anonymous — Fri, 30 Jul 2021 07:59:46 GMT

Hello,

Does the task randomly fail when executing in TMC against the Remote engine?

We see that it failed in the Google Bigquery Row component. What's query in tBigQuerySQLRow? On the big query side, did you check the activity o see what's requests being made for your execution of the job?

Best regards

Sabrina

Best regards

Sabrina

Re: Talend Job Failure - Bigquery Inserts

techwh — Fri, 30 Jul 2021 14:30:52 GMT

Hello Sabrina,

The inserts fail at both places, from local studio and ran from TMC as well.

We did check the logs from bigquery, but looks like this failure is not captured there. If the same insert query is run from Bigquery console it is running fine without any issues.

It looks like a standalone issue specific to Talend, because we do insert data using Python and it is working fine. We did reach out to Google support team and they mentioned its an issue related to Talend. Can you help to address the same.

Thank you.

Re: Talend Job Failure - Bigquery Inserts

Anonymous — Mon, 02 Aug 2021 04:25:57 GMT

Hello,

With subscription solution Talend Cloud management Platform, could you please create a support case on talend support portal so that we could give you a remote assistance(zoom meeting) through support cycle with priority.

Best regards

Sabrina