Qlik Rest Connector Pagination

[DrB1]

I'm new to using API's and I don't understand pagination enough to get all the data from Planning Center People. I connect, and can get data but depending on the settings I choose for pagination, I get anywhere from 25 to 1632 of 6507 records. I don't understand what I should be doing. The API Explorer Planning Center provides for developers (I'm a wannabe developer) gives this server response:

"root":{ 4 items

       "links":{ 2 items

                "self": "https://api.planningcenteronline.com/people/v2/people"

                "next": "https://api.planningcenteronline.com/people/v2/people?offset=25"

}

"data":[   25 items

0:{...    .................... to 25.......

"included":[] 0 items

"meta":{ 8 items

      "total_count": 6507

      "count": 25

               "next":{...}1 item

               "can_order_by":[...] 20 items

               "can_query_by":[...] 29 items

               "can_include":[...] 15 items

               "can_filter":[...] 3 items

                "parent":{...} 2 items

}

}

I interpreted this to mean I should use Pagination one of these - I've tried both and cannot get all the 6507 records:

• Offset:
• 'Start' parameter name: offset
• 'start' initial value: 0
• 'count' parameter name: 'count'
• ''count' initial value: 25
• 'total records' path: root/meta/total_count
• Next URL
  • 'next url' path: root/meta/next

I tried various "offset" settings but simply cannot get this to work. Any help anyone can provide would be most appreciated. - super indebted!!

[marksouzacosta]

Hi @DrB1,

I don't have access so far to this API, but it looks similar to one that I did before. Can you please try this on your REST API Connection? Don't worry about Offset or anything else. You should be good.

[DrB1]

@marksouzacosta It errors out - 

RestConnectorMasterTable << RestConnectorMasterTable

The following error occurred:

(Connector error: HTTP protocol error 429 (Too Many Requests): {"errors":[{"code":"429","title":"Too May Requests","detail":"Rate limit exceeded: 101 of 100 requests per 20 seconds"}]})

[marksouzacosta]

I think that indicates the pagination is working. But the number of records per call is small compared to the total number of records you have, so you are reaching the limit of requests. I found some details here: 
https://developer.planning.center/docs/#/overview/rate-limiting

Let me try to find a better way to get that...

[marksouzacosta · Accepted solution]

Can you try to add this parameter to your URL?
per_page=100
It will be something like: https://api.planningcenteronline.com/people/v2/people?per_page=100

Sorry again, I can't test myself to give you the right answer. Going blind here!

[DrB1]

Yes, @marksouzacosta !! Adding the per_page=100 to my URL worked! Thank you so much!

[marksouzacosta]

You are very welcome.