topic json files in s3 bucket to csv in Talend Studio

json files in s3 bucket to csv

Anonymous — Tue, 26 May 2020 07:47:15 GMT

Hi,

I have situation where i need to pick json files(3 files) from s3 bucket and convert one by one into csv files.

Thanks in advance.

Raghu

Re: json files in s3 bucket to csv

Anonymous — Tue, 26 May 2020 08:09:39 GMT

Hi
Use a tS3List to iterate each file and download them to local system using tS3Get.
Do the files have the same structure?

Regards
Shong

Re: json files in s3 bucket to csv

Anonymous — Tue, 26 May 2020 08:48:28 GMT

Hi Shong,

No, the files have different structure.

And also please can i have an example job for this as i am new to Talend.

Thanks & Regards

Raghu

Re: json files in s3 bucket to csv

Anonymous — Tue, 26 May 2020 08:55:31 GMT

If the files have different structure, you need to create different subjob to process each file, the job design looks like:
tS3Connection
|onsubjobok
tS3list--iterate--tS3Get
|onsubjobok
tS3Close
|onsubjobok
tFileInputJson1--main--tFileOutputDelimited1
|onsubjobok
tFileInputJson2--main--tFileOutputDelimited2
|onsubjobok
tFileInputJson3--main--tFileOutputDelimited3

Refer to the component documentation and learn these components, let me know if you any questions.

Regards
Shong

Re: json files in s3 bucket to csv

Anonymous — Tue, 26 May 2020 09:14:46 GMT

Actually i need to read the JSON files one by one and convert it into csv files and the list of csv files will be picked up tfilelist and and is loaded to redshift database which is already designed.

And also one more thing i am not able to see File option in basic settings is there any thing i need to change to get the File setting option.

Json to csv.docx

Re: json files in s3 bucket to csv

Anonymous — Wed, 27 May 2020 08:40:04 GMT

Hi
Check the 'Configure output' box and select Output type as 'CSV', then, you are able to configure the CSV options.

Regards
Shong