Loop within a loop

[MattE]

Hi, i have the requirement where i have to build and execute a number of tREST API calls, the calls are built dynamically in SQL and the number of them could change. However, the API I'm using also uses pagination so for each API call i also have to loop and call it multiple times incrementing the page number until there are no results left.

So the scenario is

Build call www.apicall1.com

loop

call page 1 www.apicall1.com&page1

call page 2 www.apicall1.com&page2

.......

Build call www.apicall2.com

loop

call page 1 www.apicall2.com&page1

call page 2 www.apicall2.com&page2

.......

and so on

I can't figure a way to do this with Talend Studio's components as tLoop has to be a start component but i essentially have to start a tLoop for each call.

Any one any ideas without resorting to Java as i'd like to make this easier to support with native Talend components?

The attached is what I've tried but because tLoop is part of the subjob it is only called once which is not the behaviour i need.

Thanks

[gjeremy1617088143]

Hi does the first call return the number of page ?

if yes you could do something like this :

first call send the value of page to a globalVar

then use a tRowgenerator with one field :

name "pageNumber"

as type String

value : String.valueOf(Numeric.sequence("s1",1,1))

in the row number field you call your globalMap :

you rely this to a tflowToiterate then you iterate on your tRest component for the loop call

and in the url you add the globalvar (String)globalMap.get("row1.pageNumber")

here row1 is the row linked to the tFlowToIterate

[MattE]

Hi @guenneguez jeremy​  thanks that would work for the pagination on one call but the issue is that once i have looped through the pages of that first call i then need to build and loop a second and maybe 3rd of 4th call and loop through their pages as well. Essentially there would be an outer loop which is where my issue lies. The ultimate goal is to combine all these into a single file.

[gjeremy1617088143]

maybe try to separate loop in distinct job for a clearer vision

[Anonymous]

yes, I agree with @guenneguez jeremy​ , loop different API calls in the main job, and do the page loop for current API in child job.

[MattE · Accepted solution]

Ok so this can't be done in one job. Thanks

[gjeremy1617088143]

it can be done in one job , but definitively i suggest you to partition this in smaller jobs to keep control of each part.