topic Re: tBigQueryBulkExec importing CSV error: You are loading data without specifying data format. in Talend Studio

tBigQueryBulkExec importing CSV error: You are loading data without specifying data format.

jamesasu — Sat, 16 Nov 2024 04:02:06 GMT

I have tried setting up a simple pipeline in Talend Studio that takes Opportunity records from Salesforce, drops them into a CSV, uploads to Google Cloud Storage and then imports into a BigQuery table. Everything works up until the BigQuery import, at which point I get the following error:

Exception in component tBigQueryBulkExec_1 (Opps_to_BigQuery)
java.lang.RuntimeException: Job failed: BigQueryError{reason=invalid, location=null, message=You are loading data without specifying data format, data will be treated as CSV format by default. If this is not what you mean, please specify data format by --source_format.}
	at sf_opps_to_bigquery.opps_to_bigquery_0_1.Opps_to_BigQuery.tBigQueryBulkExec_1Process(Opps_to_BigQuery.java:481)
	at sf_opps_to_bigquery.opps_to_bigquery_0_1.Opps_to_BigQuery.runJobInTOS(Opps_to_BigQuery.java:983)
	at sf_opps_to_bigquery.opps_to_bigquery_0_1.Opps_to_BigQuery.main(Opps_to_BigQuery.java:783)
[FATAL]: sf_opps_to_bigquery.opps_to_bigquery_0_1.Opps_to_BigQuery - tBigQueryBulkExec_1 Job failed: BigQueryError{reason=invalid, location=null, message=You are loading data without specifying data format, data will be treated as CSV format by default. If this is not what you mean, please specify data format by --source_format.}

Has anyone experienced this or know a way to get BigQuery to read from the repository schema?

Re: tBigQueryBulkExec importing CSV error: You are loading data without specifying data format.

Anonymous — Mon, 25 Nov 2019 11:54:36 GMT

Does the same thing happen if you untick the "Bulk file already exists in Google Storage" and use that mechanism to send the data?

Re: tBigQueryBulkExec importing CSV error: You are loading data without specifying data format.

jamesasu — Mon, 25 Nov 2019 22:48:57 GMT

Good thought. I've just tried that and unfortunately, that gives me the same error.

Re: tBigQueryBulkExec importing CSV error: You are loading data without specifying data format.

Anonymous — Wed, 27 Nov 2019 02:05:26 GMT

Hi,

Could you please check whether the previous tBigQueryOutputBulk is correctly pointing to the right area inside Google Storage? Especially the file extension.

There is a solution Template for Salesforce to Google BigQuery in Talend Academy. I would recommend to try the solution with the Template.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

Re: tBigQueryBulkExec importing CSV error: You are loading data without specifying data format.

jamesasu — Tue, 02 Feb 2021 05:59:20 GMT

I was able to get this working in the end by going to the Advanced Settings and checking the "Set the field delimiter" checkbox.