Qlik Community

Ask a Question

QlikView Integrations

Discussion Board for collaboration on QlikView Integration.

Announcements
Join this live chat April 6, 10AM EST - QlikView to Qlik Sense REGISTER
cancel
Showing results for 
Search instead for 
Did you mean: 
manuelaengl
Contributor II
Contributor II

Qlikview REST connector NextURL-pagination XML

 

Hi everbody,

 

we try to load data from MS Sharepoint via REST Connector 1.2.0 into a QlikView application (11.20 SR12). Pagination on Server side is set to 100 and Sharepoint uses NextURL-Pagination. The XML looks pretty much like the following (only confidential data removed):

 

<?xml version="1.0" encoding="utf-8" ?>

- <feed xml:base="http://some.sharepoint.de/xy/_api/" xmlns="http://www.w3.org/2005/Atom">

   <id>b8ac15ff-dbc5-4b78-8a8d-33d8c31b8a50</id>

   <title />

   <updated>2017-12-14T14:11:27Z</updated>

- <entry m:etag=""22"">

   <id>Web/Lists(guid'608da7f5-bf0c-4426-92de-e5cdf3e4e7f2')/Items(500)</id>

   <category term="SP.Data.AufgabenListItem" scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" />

   <link rel="edit" href="Web/Lists(guid'608da7f5-bf0c-4426-92de-e5cdf3e4e7f2')/Items(500)" />

   <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/FirstUniqueAncestorSecurableObject" type="application/atom+xml;type=entry" title="FirstUniqueAncestorSecurableObject" href="Web/Lists(guid'608da7f5-bf0c-4426-92de-e5cdf3e4e7f2')/Items(500)/FirstUniqueAncestorSecurableObject" />

   <title />

   <updated>2017-12-14T14:11:27Z</updated>

- <author>

  <name />

   </author>

- <content type="application/xml">

  - <m:properties>

     <d:FileSystemObjectType m:type="Edm.Int32">0</d:FileSystemObjectType>

     <d:Id m:type="Edm.Int32">500</d:Id>

     <d:Status>abgeschlossen</d:Status>

    </m:properties>

    </content>

   </entry>

  <link rel="next" href="http://some.sharepoint.de/xy/_api/web/Lists/getbyid('608DA7F5BF0C442692DEE5CDF3E4E7F2')/items?%24ski..." />

</feed>

 

We try to load with following settings:

 

  • Pagination Type:Next URL
  • Next URL field path: feed/link/attr:href (--> NextURL see line 24)
  • Look in header: deactivated

  

Test Connection is okay and we can load data, but QlikView loads only first 100 entries from the initial URL. And then proceeds with the script without any error message.

 

Really hope someone can help!

 

Regards Manuela

 

1 Solution

Accepted Solutions
manuelaengl
Contributor II
Contributor II
Author

Unfortunately, there is no real solution as it seems to be a bug in recognizing the XML-structure. But we found a workaround.

You can use custom pagination:

1. With a Loop you can iterate trough data packages. Starting with Initial URI

2. at the end of each select-statement you can write the next URI in a variable

3. Next.....

4. In last data package the NextHref ist empty --> Exit for

Best regards,

Manuela

View solution in original post

7 Replies
manuelaengl
Contributor II
Contributor II
Author

Unfortunately, there is no real solution as it seems to be a bug in recognizing the XML-structure. But we found a workaround.

You can use custom pagination:

1. With a Loop you can iterate trough data packages. Starting with Initial URI

2. at the end of each select-statement you can write the next URI in a variable

3. Next.....

4. In last data package the NextHref ist empty --> Exit for

Best regards,

Manuela

View solution in original post

aferrari7
Partner
Partner

Manuela, hi!

I used your script to paginate a .json page. Seem's to work fine in the beggining but it's not fetching all the records the MasterTable have to the table i need.

Coul you help me with that, please?

Sem título.png

Best regards,

Alisson

balabhaskarqlik

Did you resolved the issue, please post the solution to help fellow people.

manuelaengl
Contributor II
Contributor II
Author

Hey Alisson,

without the original data und code just can guess.. either your total is bigger than 6000 (you can increase or append with a little code first reading the total amount of records) or the pageSize is more than 100 (depends on source system).

Best regards,

Manuela

manuelaengl
Contributor II
Contributor II
Author

Hey Bala,

for the solution to the initial problem pls see above. As already mentioned, it's only a workaround but in my case that was enough.


Best regards,


Manuela

aferrari7
Partner
Partner

Manuela,

thank you for sharing your LOOP, i'm using here and works perfectly!.

But firstly i had to preload symbols count to "No Limit" so that i can have the "Root" table to paginate through the .json.


Best Regards,


Alisson


pawwy1415
Creator III
Creator III

Hi https://community.qlik.com/t5/user/viewprofilepage/user-id/50273

We are also facing the same issue. But we are trying in the Qliksense. When we tried with "/Items?$Top=300000" in the URL its fetching records up to 21000 but still some dates are missing.

By removing Top=300000 by passing any field here fetching only 100 records.

Could you please let me know how to create the Custom connection in the qliksense. The  above qlikview  connection not working in Qliksense.