topic Re: Move data from AWS S3 to EMR in Talend Studio

Move data from AWS S3 to EMR

Anonymous — Sat, 16 Nov 2024 10:16:36 GMT

hi experts,
Can anyone please help me that how can we move data from AWS S3 to EMR Cluster using talend?
Also if I have some zip files in S3 buckets, how can I unzip it using talend before moving it to EMR cluster (I assume EMR is the hadoop cluster provided by amazon)?
I am using Talend Open studio for Bigdata and running it on my local PC.
Regards
Mukesh

Re: Move data from AWS S3 to EMR

Anonymous — Fri, 04 Nov 2016 08:39:16 GMT

Hi,
If we understand your requirement very well, you can use tS3Get component to retrieve a file from Amazon S3.
The work flow should be:tS3Connection-->tS3Get(retrieve files frm s3 to local)-->tfileunarchive(unzip your file)-->EMR cluster(amazon EMR).
Best regards
Sabrina

Re: Move data from AWS S3 to EMR

Anonymous — Sat, 05 Nov 2016 15:23:03 GMT

Thanks for your reply.
Which talend component I could use for "AMR Cluster" which you had mentioned at the end ?
regards
Mukesh

Re: Move data from AWS S3 to EMR

Anonymous — Mon, 07 Nov 2016 08:06:53 GMT

Hi,
You can get Amazon EMR distribution from hadoop component.
Best regards
Sabrina