Qlik Community

Qlik Design Blog

All about product and Qlik solutions: scripting, data modeling, visual design, extensions, best practices, etc.

Employee
Employee

Working with the Qlik REST Connector, Pagination and Multiple JSON Schemas

starwars.pngRecently, I worked with a Qlik Community member to help them understand the Qlik REST Connector with Qlik Sense and QlikView. At first it appeared simple, but then he soon realized he needed to understand a bit more about how the data came back (the response), what the pagination settings were (pages of data used to retrieve more rows) and finally how to link (join, associate) other attributes that came back from the results of multiple REST API endpoints / resources. We got it all working and the results were pleasing. Needless to say were able to perform text analytics from a barrage of Facebook comments.  However, as I finalized all this in my head, I wanted to share what I've learned but in the simplest way possible. So I decided to find a very simple, publicly available RESTful service API in which I can demonstrate my findings easily. The below video presents those findings in a educational and entertaining way using the Star Wars API. Yes, that is correct, I said the Star Wars API. As a bonus, stick to the end of the video to see the Media Box Extension in action.

See this video on YouTube as well. Using the Qlik REST Connector - Pagination and Multiple JSON Schemas - YouTube

Do you know of other simple and fun, publicly available RESTful services? Share them with the Qlik Community in the comments below.

Regards,

Michael Tarallo (@mtarallo) | Twitter

Qlik

Special shout out to: Paul Hallett    (@phalt_) | Twitter - for creating an awesome resource http://swapi.co/about that allowed me to easily demonstrate the Qlik Sense REST Connector.

Resources used in this video:

Other Resources:

If using Qlik Sense Desktop please copy .qvf file to your C:\Users\<user profile>\Documents\Qlik\Sense\Apps and refresh Qlik Sense Desktop with F5. If using Qlik Sense Enterprise Server please import .qvf into your apps using the QMC - Qlik Management Console.

Disclaimer: Star Wars, the Star Wars logo, all names and pictures of Star Wars characters, vehicles and any other Star Wars related items are registered trademarks and/or copyrights of Lucasfilm Ltd., or their respective trademark and copyright holders.

18 Comments
Employee
Employee

Mike:

This is awesome!  Thanks for taking the time to put this video together!

323 Views
juc
New Contributor

Great video, Mike! Really helpful.

323 Views
kidbank3
Contributor III

I used the game of thrones api to make an app awhile ago: GoT app.qvf

It was fun and a good way to learn about the Rest connector

323 Views
Employee
Employee

Thanks for sharing the Got App Mike G.  - this great!

323 Views
Employee
Employee

Great stuff Mike.. Thanks for sharing this!

323 Views
wwi
New Contributor II

Nice one Mike!

323 Views
mstoler
Contributor II

Very nice Michael.

I am currently using the Rest Connector with pagination. I had a problem with the json format (the next field is dropped) on the last page.

My solution is to keep track of where I am and then create a new call for the last page. This works fine.

Thanks again!

Michael

323 Views
aldo-tgh
New Contributor III

Hi Mike,

Great post.

Is the ability to use "POST" (instead of "GET") available on Qlik Sense?

It will be great to have a real life example.

Thanks in advance,

Aldo.

323 Views
Not applicable

Hello Mike,

Can you tell us how to configure Rest Connector to pull data from Twitter using pagination?

Thanks!

Rima

0 Likes
323 Views
Employee
Employee

Hello Rima - since this post there has been a webinar that combines many examples - and there is a help document too:

ICYMI - Qlik Sense Tips and Tricks Webinar - Tapping into all your data

Setting up selected REST sources for data loading ‒ Qlik Connectors

0 Likes
323 Views
belias1980
New Contributor II

Hi Michael!

Thank you so much for the video, is there a step by step guide for Google Rest Connect?

I am in dire need of getting Lat/Long for my data, so i can use the map object.

Thank you

0 Likes
323 Views
Employee
Employee

Hi Basel - there isn't a detailed guide that I am aware of - but you may want to look into our Qlik GeoAnalytics option - it does all this for you.

Qlik GeoAnalytics Community

0 Likes
323 Views
belias1980
New Contributor II

Thank you Michael!

I will, i am just not sure what the cost is associated, i'll work with a rep.

But it seems that some folks were able to get a workaround for a limited use of the map object.

I can definetely see GeoAnalytics as a great option with many features.

Thank you

0 Likes
323 Views
Employee
Employee

Don't know what happened to your post, but I was able to play with the Game of Thrones (GoT) app a bit tonight and came up with some interesting insights on the death rates, characters, and houses from the data. Here's a screenshot:

2017-11-07 22_14_03-Game of Thrones REST API - Game of Thrones - Book Stats _ Sheets - Qlik Sense.png

0 Likes
323 Views
Employee
Employee

Hi Joe - I think you are referring to this one - GoT app.qvf - created by another Michael -  I added it to this reply so those may follow in case they are inquiring to what you are referencing.

323 Views
elfriede_harrer
New Contributor

Thank you so much! That is a great Video (going to show that to my son, he will love it )

323 Views
Employee
Employee

Awesome! Glad you liked it.

0 Likes
323 Views
alanmcgrath
Contributor

This is great and very easy to understand.

I've been able to get this working in QlikSense but when trying in QlikView I can't seem to get it to work with pagination.  

I've made the same selections as with the QS connector:  Pagination type = Next URL, pagination path = root/next.  Connection test works fine but when clicking select nothing happens.  If I have no pagination then when clicking select the dialog box comes up and I'm able to select the fields and pull data. (only 10 rows)  

Am I missing another step or setting for the QV version?  (I'm using QV 12.30.2 and REST Connector v1.7)

Thanks!

UPDATE:  I got it working.  So if anyone has a similar issue here is how I did it.  If you check the look in header box  in the connector dialog box and add the connection string to the script, then when clicking SELECT in edit script the select dialog will appear.   From there you can select your data and insert the script as expected.  Then you just need to modify the connection string where it says "IsNextURLFieldPathHeader=1" to "IsNextURLFieldPathHeader=0".  Seems that if you don't select that checkbox the dialog doesn't open.  Thanks!

0 Likes
19 Views