topic Re: How can I load multiple files from Google Cloud Storage into a single table in BigQuery? in Data Quality

How can I load multiple files from Google Cloud Storage into a single table in BigQuery?

Anonymous — Sat, 16 Nov 2024 07:33:24 GMT

I wonder if there is a way of bulk loading a number of files within a Google Cloud bucket into a BigQuery table since there is no tFileList component for GS in Talend?

Re: How can I load multiple files from Google Cloud Storage into a single table in BigQuery?

vapukov — Sat, 29 Sep 2018 12:09:32 GMT

You could check Google API

https://cloud.google.com/storage/docs/json_api/v1/objects/list

in Talend it would be tRESTClient

Re: How can I load multiple files from Google Cloud Storage into a single table in BigQuery?

Anonymous — Sun, 30 Sep 2018 07:32:09 GMT

Thanks for your helpful response. Are you able to guide me on how to set the tRestClient component up for this? Apart from the URL, I struggle to work out how to setup the parameters e.g authentication token so that it can talk to our google cloud or bigquery.