Skip to main content
Announcements
Join us at Qlik Connect for 3 magical days of learning, networking,and inspiration! REGISTER TODAY and save!
cancel
Showing results for 
Search instead for 
Did you mean: 
kai_1892
Contributor III
Contributor III

API parallel requests

Hi,

 

I'm looking for a solution to send concurrent requests to an API (preferably with one job only) but I'm not sure if this is possible in Talend at all. We are using Talend Open Studio.

It looks like the TRestClient component isn't capable to do this.

 

Best,

Kai 

Labels (3)
1 Solution

Accepted Solutions
Anonymous
Not applicable

This will be tough without the subscription product. You could try putting your API calls inside child jobs, passing your parameters to the child jobs using context variables, and ticking "use an independent process to run subjob". This will potentially allow them to run in parallel. It's more of a hack than a solution I'm afraid. Parallelisation is one of the pieces of functionality that is made easier with the subscription product. You should consider how many parallel calls you wish to make at one time though. A rule of thumb would be n-1 where n is the number of cores running your job.

View solution in original post

2 Replies
Anonymous
Not applicable

This will be tough without the subscription product. You could try putting your API calls inside child jobs, passing your parameters to the child jobs using context variables, and ticking "use an independent process to run subjob". This will potentially allow them to run in parallel. It's more of a hack than a solution I'm afraid. Parallelisation is one of the pieces of functionality that is made easier with the subscription product. You should consider how many parallel calls you wish to make at one time though. A rule of thumb would be n-1 where n is the number of cores running your job.

kai_1892
Contributor III
Contributor III
Author

Hi @rhall ,

thanks for your reply, however it doesn't really solve our issue, but I will try to do this with several subjobs.

Best,

Kai