Accessing the data from S3 to database table

[sushantk19]

Hello,

I am trying to access one of the S3 buckets with bucket name, access key and secret key. I want to extract the data from this s3 bucket and load to database table. Can someone share sample ETL job design along with configurations if someone has done this before? what components do we use?

[manodwhb]

@Sushant Kapoor​ , I can not able to share the job screenshot. but below are the components I used.

tPreJob-->tS3Connection -->On Component OK -->tDBConnection

main job

tS3Get-->On Component OK --> tFileinputDelimitted-->tDBOutput

tPostJob-->tS3Close-->tDBClose

Thanks,

Manohar

[manodwhb]

@Sushant Kapoor​ ,check the below link will help you.

https://www.nuwavesolutions.com/using-talend-to-extract-data-from-an-aws-s3-bucket-to-postgres/

Thanks,

Manohar

[sushantk19]

@Manohar B​ : Thanks. I will check and let you know.

[sushantk19]

@Manohar B​ : do you know if we can access/copy all the files from a folder. they are gzip files

in "Key" field what do we give to access multiple files.please see the screenshot.

If i give *.*, it gives me an error.

[sushantk19]

@manodwhb: how can we access(get) multiple files from S3 bucket to our local folder as i have to unzip the files also.