topic Re: Read .json file from S3 with multiple sub folders in Talend Studio

Read .json file from S3 with multiple sub folders

SR2210 — Sat, 16 Nov 2024 04:20:33 GMT

Hi,

Does anyone know how to read json file from S3 that is segregated into different subfolders?

My json file's path is in this format: year/month/day/text.json

For example;

"/2018/12/1/text1.json"

"/2018/12/2/text2.json"

"/2018/12/3/text3.json"

Thank you in advance!

Re: Read .json file from S3 with multiple sub folders

Anonymous — Mon, 21 Oct 2019 05:42:56 GMT

Hello,

Could you please let us know if this component tS3Get can meet your needs?

Here is online component reference:

https://help.talend.com/reader/MEJTft6Phcbzeotwjs0Ivw/AtIpnMYkEW9Flmg8VWPO3Q

Best regards

Sabrina

Re: Read .json file from S3 with multiple sub folders

Anonymous — Mon, 21 Oct 2019 16:12:17 GMT

Hi,

If you know the list of dates prior to data extraction, you can make it an input to tFlowtoIterate where you will send the parsed values of each date to identify the directory. Then you can connect to tS3Get to pick the data for each date. In short, the input dates can be from a file or DB and the high level components required are as below.

tFileInputDelimited -> tMap->tFlowtoIterate-(Iterate)-> tS3Get

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved