topic Re: Run a task for multiple times simultaneously through job conductor by using a rest API call in Talend Studio

Run a task for multiple times simultaneously through job conductor by using a rest API call

KarthikGs — Sat, 16 Nov 2024 08:24:48 GMT

Hi team,

I'm using enterprise version talend big data 6.4 version. I have a use case where i need run a job from TAC for multiple times simultaneously without creating the copies of the job in the job conductor.

Here is my exact problem statement,

I have a job that basically copies contents of a file in to a database, here the file path will be changing dynamically so i set it up as a context variable.
Now i have a airflow dag which calls my above talend job using a API call to trigger the job with the proper contexts. However, the problem is that if I trigger that job via API call to TAC, I have to wait for it to end before I call it with a different file path.
I want the job to trigger, then detach from the TAC immediately so I can call the same job with different contexts variables and have it run simultaneously with the first call, instead of having to wait for the first call to finish.
is there any way to solve this puzzle in TAC itself as we cannot alter the developed talend package?

Any help is appreciated.

P.S: I have come across a similar post in the community but it is unresolved.

https://community.talend.com/t5/Design-and-Development/Running-same-job-multiple-times-for-reading-file-from-same/m-p/67404

Thanks in advance.

Re: Run a task for multiple times simultaneously through job conductor by using a rest API call

Anonymous — Fri, 13 Apr 2018 07:48:03 GMT

Hi Karthik,

If you have runtime server then we can run Talend job through rest API. So in this case you have to create a REST API by using Talend ESB and deploy it in runtime server.

Re: Run a task for multiple times simultaneously through job conductor by using a rest API call

KarthikGs — Fri, 13 Apr 2018 08:31:16 GMT

hi @uganesh we dont have talend ESB in our stack.

Re: Run a task for multiple times simultaneously through job conductor by using a rest API call

KarthikGs — Wed, 25 Apr 2018 07:06:53 GMT

hi ravi, can you elaborate your solution. or else you can guide me to any valid link with the solution for the proposed problem statement

Re: Run a task for multiple times simultaneously through job conductor by using a rest API call

ak93 — Fri, 04 Oct 2019 15:49:30 GMT

we are running into the same issue when using talend cloud how can we trigger the same job from external scheduler . the error we get is task is already running when the first job is running and we rerun the same job with different parameters.