topic Re: Multiple paginated APIs in one job in Talend Studio

Multiple paginated APIs in one job

lizzy1 — Wed, 16 Oct 2019 13:06:58 GMT

I have a job in mind that I'm not sure how to execute. I need to call two separate REST APIs then join their outputs before loading to my database. The problem is that both of these API's only return a few records per page and need a page parameter for subsequent calls. When I have done similar jobs with only one API call, I have used a flow meter to tell when I have reached the last page. Is it possible to have two separate pairs of tFlowMeter and tFlowMeterCatcher in the same job, without interfering with one another? If not, how should I accomplish this?

Re: Multiple paginated APIs in one job

root — Wed, 16 Oct 2019 14:21:43 GMT

Maybe try creating two sub jobs and calling them from one parent job?
OR
use tIterate component after fetching total records and then call the remaining records?

Never worked with tFlowMeter

Re: Multiple paginated APIs in one job

lizzy1 — Wed, 16 Oct 2019 14:47:49 GMT

I used your idea of making separate child jobs, and it worked wonderfully.