topic Re: Load data from S3 to PostGres in Talend Studio

Load data from S3 to PostGres

Anonymous — Thu, 13 Oct 2016 22:23:21 GMT

Hi,
I would like to know a sample pipeline to load data from S3 to PostGres and MySql servers.
Thanks

Re: Load data from S3 to PostGres

Anonymous — Mon, 17 Oct 2016 08:19:49 GMT

Hi,
Here are tS3XXX and tPostgresqlXXX components in talend.
You can use ts3get component to retrieve data from Amazon s3 and then load them into postgresql DB.
The work flow should be: tS3xxx-->tpostgresqloutput
-->tmysqloutput
Best regards
Sabrina

Re: Load data from S3 to PostGres

Anonymous — Mon, 17 Oct 2016 17:58:44 GMT

Thank you Sabrina,
This is useful, one query is that using ts3put , do we need to write it to a file server before we use tpostgresqloutput / tmysqloutput or would tpostgresqloutput/tmysqloutput be able to consume it directly from ts3put.
Regards,
Srivignesh KN

Re: Load data from S3 to PostGres

Anonymous — Mon, 17 Oct 2016 22:08:58 GMT

Hi Sabrina,
Thanks for the reply.
My use case is to load data from S3 to PostGres and i am trying to read an existing file from S3 bucket and load it into PostGres DB.
However, i see the description of the tS3_Put connector is to load data into S3.
The connector also asks for the file name to be uploaded as a mandated input (Highlighted in the Picture). I would like to understand how we would be able to read file from an existing bucket from S3 and load to PostGres/Mysql.
Thank you for your time on this, please do let me know if you require any additional details.
Regards,
Srivignesh KN

Re: Load data from S3 to PostGres

Anonymous — Thu, 20 Oct 2016 08:52:31 GMT

Hi,
You need to use tS3Get component in your use case which is used to retrieve a file from Amazon S3.
https://help.talend.com/search/all?query=tS3Get&content-lang=en

Best regards
Sabrina