Rest Connector - Pagination using Next URL

Fri, 15 Jul 2016 07:28:09 GMT

I'm trying to get data from the DOAJ api (www.doaj.org) using the REST Connector. But I can't get the pagination right, I only get the first page. I'm using this request url: https://doaj.org/api/v1/search/journals/*?page=1&pageSize=100

At the end of the result from this request there is the "next" field so I'm assuming that Next URL is the pagination type to use but I don't know what to put in the "Next url field path". And do I also need to fill in the fields for "Query parameters" and "Query headers"?

Sincerely

Henrik

Re: Rest Connector - Pagination using Next URL

Anonymous — Fri, 07 Apr 2017 15:27:37 GMT

You can use the custom option and use a loop to walk through the next pages:

Let total = Round(FieldValue('X-Total-Count',1)/100);

Let startAt = 1;

for startAt = 1 to total step

WITH CONNECTION(Url "https://doaj.org/api/v1/search/journals/page=$(startAt)");

topic Rest Connector - Pagination using Next URL in Connectivity & Data Prep

Rest Connector - Pagination using Next URL

Re: Rest Connector - Pagination using Next URL