Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
I have a file that lands in AWS S3 several times a day. I am using Talend as my ETL tool to populate a warehouse in Snowflake and need it to watch for the file to trigger my job. I've tried tWaitForFile but can't seem to get it to connect to S3. Has anyone done this before?
Waiting for a file to land means you're in event mode, so real time/ESB mode. With a route and a cAWSS3 you might be able to listen to this folder.
If you don't have ESB, using ETL you have to schedule the job every 5 minutes for example, connect to s3, download file if present, if not stop the job. The next 5 minute the job will do the same process
If the file that lands on S3 is not scheduled there is no other way for you to wait for an event using ETL