Need info about tBigQueryOutput error

[Anonymous]

Hi,

I wish to load a table into BigQuery and am using tbigqueryoutput component to do the same. I thought, all that required was the dataset and the table name where it should be loaded and the data that is coming from talend.

But I see there are 2 other different parameters that are requested for

1. Local File Name

2. File on Google Storage Configuration.

My queries are

1. Why would there must be a local file name storing locally the output file to be pushed into bigquery? What do I have to do, if I do not wish to store any such details locally at all.

 

2. What content does the Google storage configuration exactly contain? Where is this file created?

Because with out this file the loading is not happening. Where in Big Query do I have to put the details of the google storage and what kind of details should exactly be containing into it.

Lastly,

I tried pushing records from talend into a Bigquery, but it keeps asking for the bucket details. Can anyone of you explain what kind of details should be present in the gs://<<bucket_Name>>/<filename_to_be_used_in_talend>>. PFB the screenshot of what parameters do i really mean.

[Anonymous]

Hi tsreenath,

this components put your local file in a bucket in google cloud storage and then import into bigquery table.

API Gloud storage and bucket must be activated and created.

It's clear ?

[Anonymous]

Do you mean Cloud Storage JSON API?

Also do we require any other APIs such as BigQuery API, Cloud Storage Service, Cloud Data Store API for loading the data from Talend into Talend BigQuery?

[Anonymous]

You need BigQuery & Google Cloud Storage

And for acccess, we need Id client + access key

For Cloud Storage, go to GCP Console and generate key

https://console.cloud.google.com/storage/settings?project=YOUR_PROJECT

[Anonymous]

I already have the following details
Connection Details
1. Client ID
2. Client Secret
3. Project ID
4. Authorization Code

Google Storage Configuration
1. Access Key
2. Secret Key
3. Bucket


And I can able to extract the data from Google BigQuery (using tbigqueryinput component). Only problem I face is while loading the data into Big Query (i.e. using tbigqueryoutput component).



I do not know what exactly the Google storage file mean and what are required from Google cloud side (any api/access) to load the data into bigquery from Talend successfully. Can any of you confirm me any api or service required to get enable for loading bigquery through Talend.

[Anonymous]

Google Storage is used to copy file in Google before to load in Bigquery. The job load file from Google Cloud storage and not your file in local.

We need only Google Cloud Storage api.

Do you test to run your job, do you have error ?

[Anonymous]

Hi,

PFB the actual error which I am getting, can you please tell me what exactly is needed to be done to get this error resolved. With this message, Can I, in anyways assume the Google Cloud Storage API is already available or not?



[cid:image001.png@01D2DA34.5EA8EFD0]

[Anonymous]

Can you re-upload your image ?

[Anonymous]

can you add a print screen for your bucket ?

[Anonymous]

why file has not the same file
In your bucket CurriculumTemplate.json, in the error talend curriculum_template.json