Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi,
I try to use the Rest Connector pagination
After load I get in table 8475 lines
But in this tables I get not distinct lines ( only 25 distinct lines)
Hello Maryna,
I had also a lot of trouble using the pagination in REST API.
I had a similar problem :
I am using REST API with JIRA.
I used Pagination : None and the pagination was managed in the script.
The code was :
// Implement the logic to retrieve the total records from the REST source and assign to the 'total'
// Getting the "total" value from the first API URL call
totalTable:
SQL SELECT
"total" AS "total_u1"
FROM JSON (wrap on) "root";
//This variable is a total of JIRA entries. Will be used later in 'For' loop
LET total = Peek('total_u1',0,'totalTable');
DROP Table totalTable;
LET totalfetched = 0;
LET startAt = 0;
LET pageSize = 999;
FOR startAt = 0 to total
TRACE "startAt: " $(startAt);
RestConnectorMasterTable:
SQL SELECT
"expand" AS "expand_u0",
"startAt",
"maxResults",
"total" AS "total_u1",
"__KEY_root",
(SELECT
...
...
FROM JSON (wrap on) "root" PK "__KEY_root"
WITH CONNECTION(
QUERY "startAt" "$(startAt)"
);
// Change URL included in 'WITH CONNECTION' as needed to support pagination for the REST source.
// Please see the documentation for "Loading paged data."
startAt = startAt + pageSize;
NEXT startAt;
all the pages have been read, but on each page the identifiers are repeated.
So instead of having the identifiers from 1 to 5000, I have 5 times 1, 5 times 1.... 5 times 1000.
The solution was to use Pagination "Offset":
What is really strange, this solution works on our server.
Bur when I try to use this solution on my desktop, it does not get all the JIRA issues (if gets only 9400 out of 46000 ?!!!!).
And the best part is : The solution with pagination "None" and the scripting works fine.
Maryna, you could maybe try out the solution with the scripting...
