I've the following job wich works ok: tREST_1 > tXMLMap_1 > tExtractJSONFields_1 > tLog
The problem I'm facing is that the tRest URL will return a body in which 100 records are returned (in the data section) but also a new URL (in the pagination section) to retrieve the next 100 records (if resultset contains more then 100 records). How can I loop this so I retrieve all the records I'm looking for?
The order of the job should be:
retrieve data of first tRest (tExtractJSONFields Loop Jsonpath query: "$.data
"
extract the fields
evaluate if body contains column "next" in the pagination section (tExtractJSONFields Loop Jsonpath query: "$.pagination"
if true load this URL and retrieve the next 100 records
loop while pagination section contains column "next"
This isn't exactly what you want to do, but it is close enough. I have written a tutorial on getting spotify data from Facebook. This requires pagination and looping through REST calls. The tutorial is
here. The section you want starts with the title "The
GetMySpotifyListeningHistory Job".
This isn't exactly what you want to do, but it is close enough. I have written a tutorial on getting spotify data from Facebook. This requires pagination and looping through REST calls. The tutorial is
here. The section you want starts with the title "The
GetMySpotifyListeningHistory Job".
