AWS s3 Filewatcher

[Anonymous]

Hi,

How to implement S3 filewatcher in talend..?

Thanks.

[vapukov]

do You want real watcher? ("real-time")

or just want from time to time check file exists or not?

[vapukov]

I suggest You check S3 notification system http://docs.aws.amazon.com/AmazonS3/latest/dev/ways-to-add-notification-config-to-bucket.html

I not test this way, but should be work - S3 -> SQS -> Talend 

[Anonymous]

just want from time to time check file exists or not?

[vapukov]

tS3BucketExists - if need check Bucket

tS3List - uncheck "List all object" checkbox and test patterns for objects

[Anonymous]

Hi  ,

I just want to check weather file is exist in s3 bukect or not around one hour means i thought to use the tsleep and tloop and iterate around one hour if file is not exist after one hour then we will populate the message file is not exist.. 

[vapukov]

You can use this components

Also - You can avoid tLoop and tSleep - just run You Job by scheduler. No reason to keep memory when it not necessary.

Other way (I use them some time) - install AWS cli on Talend Machine, and do this with command line interface with tSystem.

Example - I use this for fetch most resent backup from S3.

[Anonymous]

Other way (I use them some time) - install AWS cli on Talend Machine, and do this with command line interface with tSystem.

Example - I use this for fetch most resent backup from S3.

Hi ,

how to install  AWS cli on Talend Machine..

THanks.

[vapukov]

they have excellent docs on site, all what you need - keys which will need install when setup client
for this reason better to create separate for Talend

http://docs.aws.amazon.com/cli/latest/userguide/installing.html

http://docs.aws.amazon.com/cli/latest/userguide/installing.html