topic Re: event based trigger in talend cloud in Talend Studio

event based trigger in talend cloud

nivedhitha — Sat, 16 Nov 2024 05:22:49 GMT

Hi,

I have been using the talend subscription version( talend cloud). I now have a requirement where my job has to be triggered as soon as a file is available in amazon s3. So my question is does talend cloud have support for event based triggers? If not how do you suggest implementing this?

thanks in advance

Re: event based trigger in talend cloud

Anonymous — Thu, 04 Jul 2019 23:15:39 GMT

@nivedhitha

Please refer the link given in the below KB article.

https://community.talend.com/t5/Design-and-Development/To-trigger-talend-Job-based-on-S3-bucket-event-notifications/td-p/31905

The link is for TAC but with some modifications, you should be able to do it in TMC also.

Below article shows the similar requirement for Azure flavour.

https://community.talend.com/t5/Architecture-Best-Practices-and/Using-Azure-Functions-to-trigger-Talend-Cloud-Jobs/ta-p/124225

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

Re: event based trigger in talend cloud

nivedhitha — Sun, 14 Jul 2019 22:48:15 GMT

@nthampi ,

I have set up s3 to send notifications to SQS and my Talend Job is also able to receive messages from the queue , parse the JSON and get the name of the file uploaded in s3.

Im looking for help/suggestion on how to trigger this job(which will eventually be deployed on talend cloud) as soon as a file becomes available on s3. Is it possible only by involving lambda in between?

thanks in advance

Re: event based trigger in talend cloud

Anonymous — Sun, 14 Jul 2019 23:00:37 GMT

Hi,

Lambda is the best triggering mechanism in your case. End of day you are trying to build an event based triggering mechanism and Lambda is a service whose primary purpose is to handle this situation.

Unfortunately I will not be able to check the details of the solution right now. I would suggest to get the help of Talend professional services in this case since its a one off setup operation.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂