topic Re: Archive File from ftp to S3 in Talend Studio

Archive File from ftp to S3

Anonymous — Sat, 16 Nov 2024 07:55:10 GMT

Hi,

I am new to Talend Data Fabric. We have requirement to archive files from ftp to Amazon S3. Please help me on this.

Thanks,

Aparna

Re: Archive File from ftp to S3

Anonymous — Mon, 23 Jul 2018 03:22:46 GMT

Hello,

From your job requirement, the work flow looks like:

tFTPConnection-->tFTPGet-->tFileList-->tFileInputXXX-->tS3Output

Please take a look at component references:TalendHelpCenter:FTP components and

TalendHelpCenter:Amazon S3 components

Let us know if it is OK with you.

Best regards

Sabrina

Re: Archive File from ftp to S3

Anonymous — Thu, 26 Jul 2018 06:50:48 GMT

Hi Sabrina,

Actually my requirement is to copy files(as in lot of files) from FTP server and directly it to S3 and also log the time taken.

This is to be scheduled every 30 minutes.

And tS3output is not avaliable in my project Talend Data Fabric.

So please help.

Regards,

Aparna

Re: Archive File from ftp to S3

Anonymous — Thu, 26 Jul 2018 08:24:52 GMT

Hi Aparna
The job design looks like:
tFTPFileList--iterate--tFTPGet--oncomponentok--tS3put--oncomponentok--tFileDelete
tftpget: download the file to local system.
tS3put: put the file to S3 server from local system.
tFileDelete: delete the file from local system if needed.

Please take a look at the component documentation and learn each component mentioned here.

Regards
Shong

Re: Archive File from ftp to S3

Anonymous — Thu, 26 Jul 2018 14:43:28 GMT

Hi Shong,

Thanks for your reply.

Is it possible to directly copy file from SFTP to S3 without placing it on local system?

Regards,

Aparna

Re: Archive File from ftp to S3

Anonymous — Fri, 27 Jul 2018 03:50:31 GMT

Hello Aparna,

So far, talend don't support for transferring data by air. You have to place your ftp files on local system firstly and then load them to S3.

Best regards

Sabrina

Re: Archive File from ftp to S3

Anonymous — Fri, 27 Jul 2018 08:51:55 GMT

Hallo Aparna, I believe this scenario is possible with an ESB Mediation Route.

Can you design a Route as below and see if that works

cFtp -> cAwsS3