topic Re: run Talend DI job on live data in Talend Studio

run Talend DI job on live data

Anonymous — Sat, 16 Nov 2024 09:07:32 GMT

I'm using Talend 6.2 enterprise version, and have a job which takes a data from Rest api and process further.

I'm looking for something live listening of rest API. Is that possible I could make my job ever running so that whenever new data arrives on api server, we will be keep consuming data through rest API. Some sort of live streaming.

Any suggestions please..!!

Re: run Talend DI job on live data

Anonymous — Sun, 29 Oct 2017 10:31:25 GMT

Hi

The Rest API server generally does not provide a mechanism for listening, so it is impossible to listening the API server with these tRest, tRestClient or tHttpRequest components, I have a similar case and I schedule the job to run regularly to retrieve the newest data.

Regards

Shong

Re: run Talend DI job on live data

Anonymous — Sun, 29 Oct 2017 10:43:03 GMT

I'm using trestrequest and trestresponse components in Talend DI.

I have a set up where whenever user subscribe to a website that information we are getting from rest api and sending an mail to them using another api.

How can I achieve this for ever running process ? I mean for real time processing, I want my job to be keep running and keep getting the information from api when ever user subscribe the website.

Is there any way to achieve this ?

Re: run Talend DI job on live data

Anonymous — Sun, 29 Oct 2017 11:17:50 GMT

Hi Kate,

This is very much what Talend ESB is designed for, whereas DI is intended for one-off/periodically scheduled jobs which start, do their work, and then stop.

With ESB, you create services which run on your server, and provide end-points which a remote system, like your website, can connect to in the manner you're describing. This triggers the relevant processing, which is carried out immediately, and at a minimum will return at least confirmation that everything has been completed successfully.

Regards,

Chris

Re: run Talend DI job on live data

Anonymous — Sun, 29 Oct 2017 11:40:03 GMT

Hi Chris,

Do you mean to say the job which I'm having right now in Talend DI, move that to esb ? If not, Can you please give some example what type of service and how can I create that in ESB ? And more over how can I link that to Talend DI job for further processing ?

Second, How to deploy this Talend ESB service with Talend DI job together in Tac ?

Re: run Talend DI job on live data

Anonymous — Sun, 29 Oct 2017 13:02:31 GMT

Hi Kate,

You'll be able to re-implement the core processing etc. from your current DI job in ESB, as it includes the same components, plus a good number of additional ones, and Studio features which are necessary for developing services.

There is a lot more involved when developing services, compared to creating DI jobs, so I'd recommend working through the ESB Getting Started Guide, which covers the basics you'll need to accomplish your current goals:

https://www.talend.com/products/application-integration/esb-manuals-release-notes/#product_user_manuals_pdf

You can import jobs which have been exported from DI into ESB.

Regards,

Chris