tBigQueryInput DML Update BigQuery

[geloofy]

Hi

I'm running a big query update DML command using the tBigQueryInput component.

The command executes successfully but the component doesn't stop running or move onto the next task. It's as if it is waiting for a resultset from the DML command however there is no result because it is an UPDATE.

Does anyone have experience with this?

[Anonymous]

Hello,

Could you please indicate on which build version you got this issue? What does your job design look like? Screenshots will be preferred.

Best regards

Sabrina

[geloofy]

Hi Sabrina

I'm using Talend Open Studio for Big Data Version: 6.5.1.20180116_1512

tBigQueryInput executes a DML statement, eg. delete from table where 1=1

This runs against BQ and deletes all data in a table but tBigQueryInput doesn't finish. I have to use tJavaRow to kill the process using this piece of code: System.exit(99);

[Anonymous]

Hello,

Could you please try to use tBigQueryInput with a Query, and include: "#standardSQL\n" then the query you wish to execute to see if it works?

Here are some links for reference:

https://cloud.google.com/bigquery/docs/updating-data
https://cloud.google.com/bigquery/docs/reference/standard-sql/

Best regards

Sabrina

[geloofy]

Hi Sabrina

Sorry I didn't mention that I am already executing the commands using "#standardSQL\n".

Unfortunately I have the same issue with or without the "#standardSQL\n".

Regards

Geoff

[Anonymous]

Hello,

Would you mind posting your tBigQueryInput DML statement here?

Best regards

Sabrina

[geloofy]

Hi Sabrina

Here is an example of a DML that I'm running:

"#standardSQL\r\n
INSERT INTO `sdk.app` (_id,package_name)
SELECT S.*
FROM `sdk.appDelta` AS S
LEFT JOIN `sdk.app` AS D
ON S._id = D._id
WHERE D._id IS NULL"

[Anonymous]

Hello,

It seems that this is something that the Spark API doesn't offer yet. We will make an investigation on this issue to see if there is any workaround and will keep you posted.

Best regards

Sabrina

[geloofy]

Thank you! Hope to hear back from you soon.

[Anonymous]

Hello,

Here exists a new feature jira issue:https://jira.talendforge.org/browse/TBD-3547.

Feel free to vote for it.

Best regards

Sabrina