Pagination with Multiple @odata.nextLink Values in Qlik REST Connector

jmcy2408 — Sat, 08 Mar 2025 00:46:24 GMT

Hello Qlik Community,

I'm working on a REST API integration with Microsoft Dynamics Business Central where I'm using a batch transaction request to pull multiple company data in a single query for efficiency. However, I’m running into an issue with pagination in the Qlik REST Connector.

The API returns multiple responses in a single call, and each response contains its own @odata.nextLink for pagination. I’ve tried different variations of the Next URL Path, but so far, I haven’t been able to successfully paginate through all the responses.

Here’s an example of the API response structure:

{ "responses": [ { "body": { "data": [...], "@odata.nextLink": "https://api.businesscentral.dynamics.com/tenant/api/v2.0/companies(abe78fd7-84ab-ec11-80f1-000d3a0d0053)/generalLedgerEntries?aid=FIN&$skiptoken=900f4805-4147-ee11-be6d-6045bdd2c536" } }, { "body": { "data": [...], "@odata.nextLink": "https://api.businesscentral.dynamics.com/tenant/api/v2.0/companies(bcfcf5d3-4c90-4dff-9b29-63d2fb95a173)/generalLedgerEntries?aid=FIN&$skiptoken=d235fd2f-dc57-4186-8a67-e9430b445d7c" } } ] }

As you can see, each response in the responses array has its own @odata.nextLink, which points to the next set of data for each respective company. The URL format looks like:

https://api.businesscentral.dynamics.com/<TENANT>/api/v2.0/companies(<COMPANY_ID>)/generalLedgerEntries?aid=FIN&$skiptoken=<TOKEN>

Where <TENANT> is the tenant identifier, <COMPANY_ID> is a unique company ID for each company in the request, and <TOKEN> is a dynamic pagination token.

My Question:

Does the Qlik REST Connector support multiple iterations of the @odata.nextLink (one for each response object) during pagination in a batch transaction request?
If so, how do I correctly set the Next URL Path to iterate over each @odata.nextLink in the responses array?

I’ve tried the following paths without success:

root/responses/body/@odata.nextLink
root/responses/0/body/@odata.nextLink
root/responses/*/body/@odata.nextLink (which I hoped would loop through each response)

I can get this working in Python, but I’d much rather have it built directly in Qlik. The workaround I've developed is running it in Python, converting the results to a QVD, and then uploading that into my Qlik Cloud tenant. But ideally, I want to streamline this process and avoid the need to rely on Python.

Has anyone encountered a similar issue or have any suggestions on how to approach this?

Any guidance or tips would be greatly appreciated!

Thanks in advance!

Re: Pagination with Multiple @odata.nextLink Values in Qlik REST Connector

marksouzacosta — Mon, 10 Mar 2025 03:14:20 GMT

Hi @jmcy2408 ,

I don't think Qlik REST Connectors supports such thing. Actually, I never saw anything like this before - and I work with APIs for many years.

One possible solution for that is to use While Loops that checks if nextLink is not empty together with some For Loops. Some short of recursive approach may be needed too depending on the responses from the NextLink of the first level Response.

Regards,

Mark Costa

Re: Pagination with Multiple @odata.nextLink Values in Qlik REST Connector

jmcy2408 — Mon, 10 Mar 2025 19:32:49 GMT

Thanks for the suggestion, thats what i've implemented in Qlik and whilst it works - its still not the greatest. I would had hoped by now that Qlik would have a connector for Dynamics Business Central.

Think i'm going to progress with Python for data extraction and have this run on schedule on a VM and upload as QVD to our cloud for onward use - this seems to be the easiest and most scalable option, as it allows me to run multiple API calls in parallel.

topic Re: Pagination with Multiple @odata.nextLink Values in Qlik REST Connector in Integration, Extension & APIs

Pagination with Multiple @odata.nextLink Values in Qlik REST Connector

My Question:

Re: Pagination with Multiple @odata.nextLink Values in Qlik REST Connector

Re: Pagination with Multiple @odata.nextLink Values in Qlik REST Connector