REST API for Freshdesk not Paginating when being used as a data connector
I am almost certainly doing something bone-headed here, but rather than continuing to bash my head against the proverbial brick wall, I decided to come here and show my ignorance for all to see, in the hope of getting a rapid (and correct) answer.
What I am trying to do is use the REST Web Connectors in Sense to communicate with our Freshdesk instance, and paginate the results, to ensure that we always have the correct information coming through.
According to the Freshdesk API guide (here), there is a field in their response, called "link", which will populate with the Next Page URL when another page exists - I can confirm that this part seems to be true:
Now, i'm not selecting it in my code, as there is literally nothing else in the headers that I am remotely interested in, but no matter what I put in the data connector pagination area, I always get the same number of items coming through: one page worth (there are at least 5 pages worth of items). My best guess, is that I need to select the link field somewhere in my data load editor, then simply drop it once it is used, but I have no clue how to accomplish this. Here is what I have in the data connector currently - but please note: this does not paginate. Neither does the alternative setting where I put _response_header\link.