'Insert Overwrite' into a partition in hive table equilant functionality in Bigquery?

Anonymous — Sat, 16 Nov 2024 06:46:09 GMT

Hi Everyone,

We are in the process of migrating from hdfs/hive to GCS/Bigquery infrastructer and using Talend v6.3 bigdata edition for ETL logic.Most of the hive tables are being partitioned and while loading we are using 'INSERT OVERWRITE' syntax in hiverow component while inserting data into a perticular partion.

Now we need to replace hive with bigquery table with same metadata and partition details.I dont see option INSERTOverwrite in Bigquery!

Could someone help me on how do i implement similar 'InsertOverwrite Logiv' in Bigquery which is currently we are doing in Hive?

Thanks in advance and much appreciated any suggestion.

Sam

Re: 'Insert Overwrite' into a partition in hive table equilant functionality in Bigquery?

vapukov — Fri, 25 Jan 2019 02:11:09 GMT

no features in Talend component for this, but as a variant, you could use command line client for this:

https://cloud.google.com/bigquery/docs/bq-command-line-tool

https://cloud.google.com/bigquery/docs/reference/standard-sql/dml-syntax

https://stackoverflow.com/questions/51079803/bq-command-line-for-dml

topic 'Insert Overwrite' into a partition in hive table equilant functionality in Bigquery? in Talend Studio

'Insert Overwrite' into a partition in hive table equilant functionality in Bigquery?

Re: 'Insert Overwrite' into a partition in hive table equilant functionality in Bigquery?