How to extract data from s3 folder by folder in a single query

[Anonymous]

Hi Team,

I have a requirement like this to extract data from s3 bucket to PostgreSQL table

For Ex: 

In s3 bucket - i have folder like 

PI/Data/2019-10-18/  .json files

            /2019-18-19/ .json files

           /2019-18-20/ .json files

How can i achieve this ? Once it finish  loading the 2019-10-18 .json files to PostgreSQL tables , it should automatically go to the next date folder and send those as well to PostgreSQL tables.

Thanks,

Prabuj

[Anonymous]

Hi @prabuj27 ,

You can try using tS3Filelist to get all the files from S3 and use the key prefix as "*.json" and load into PostgreSQL table.