Esri AcrGIS integration with Qlik GeoAnalytics

[Nabeel_Asif]

Esri AcrGIS Online integration with Qlik GeoAnalytics

Introduction

Qlik GeoAnalytics gives us the power to perform spatial analysis within Qlik Sense and QlikView apps. While many Qlik customers have existing GIS tools, their spatial assets are quite often sitting in silos. Combining spatial data with other data through Qlik's associative engine greatly enhances the business value that can be derived from such assets.

Some BI tools can make use of local files such as KML or Esri shape files as data sources, but this introduces security and maintenance headaches. With Qlik GeoAnalytics we have the capability to consume spatial content via hosted services. This document gives a step by step guide on how we can load data from secure content in ArcGIS to Qlik Sense. 

The process described here is similar for for ArcGIS Online and on-premise, but I have only tried it out with ArcGIS Online.

The integration can be broken down into these steps:

1. Implement App Login in ArcGIS
2. Setup Qlik to get a security token
3. Build a query for the secure content
4. Load the data using Qlik GeoAnalytics

If the content is public, the process is simpler and you can skip the first two steps and ignore all the talk about tokens.

To try out these steps you will need an ArcGIS Online developer account which you can get here.

Implement App Login in ArcGIS

ArcGIS provides a way for applications to use secure content by using App Login. Once you register for a developer account, you'll see the option to create a new Application. Do that and you will be able to get the stuff we need to generate tokens in Qlik.

You'll need the Client ID and Client Secret to automate the process of getting new tokens in Qlik.

Setup Qlik to get a security token

We can use the Qlik REST connector to obtain a token in the load script. There is a good explanation of the process on the ArcGIS side here.

For ArcGIS online we need to make a request to: https://www.arcgis.com/sharing/rest/oauth2/token/

For an on-premise server the url will look like this: https://<host>:<port>/<subdirectory>/sharing/rest/oauth2/token

Create a new REST connection in Qlik Sense with the following parameters and the Client ID and Client Secret you obtained earlier.

This will generate a load script to obtain the 'access_token'. Store the token into a variable in the load script. This can be done with the code below.

// Store the token in a variable that can be used as a request parameter

LET vToken = Peek('access_token', 0, 'root');

Tokens expire after a given period, but with this setup we can get the load script to obtain a new token before loading data from ArcGIS.

Run the load to confirm that you're able to obtain a token. Copy the token text as we'll need that to build a query. One way of doing this is to export it from a table object in Qlik Sense.

Build a query for the secure content

You'll need some private content on ArcGIS Online to test out this bit. Create a new layer by uploading some spatial content such as a zipped shape file.

Make sure the security is set as private.

Once the layer is created you will be able to get a Service URL.

You can validate that the layer is private by trying out this URL in your browser. You should see a 'Token Required' message.

You can add the token we obtained earlier to the URL manually using the syntax: <Service URL>?token=<token>

This lets you get to the Service Directory. From here click on the layer name, scroll all the way to the bottom and click on Query.

You now need to build a query for this layer. I've highlighted the most common parameters you might need to set for the query. Once you're done click on the Query (GET) button.

If everything goes well you should see the JSON output of the query in your browser. Copy the URL from your browser's address bar as we will use this for loading the layer in Qlik GeoAnalytics.

Load the data using Qlik GeoAnalytics

We can now use the URL we obtained for the query to load the layer into Qlik Sense. If you are loading a polygon layer I would recommend using the Simplify operation in the GeoAnalytics connector and excluding the original geometry field in the load. If you are loading a point layer you can use the Load operation.

Once GeoAnalytics has generated the script you can change the hard coded token with the variable using dollar sign expansion i.e. $(vToken).

You should now be able to load the secure ArcGIS content into your Qlik Sense app.

[Anonymous]

Hey Nabeel, Thanks for the documentation. I'm trying it out and getting stuck at the step for Loading data using Qlik GeoAnalytics. When I use GeoAnalytics from the Data Load Editor I'm pasting the JSON from the ArcGIS REST point for the private layer in Custom server address. When I select Save and try to Select Data a loading message flashes for a second then disappears with no result.

Any tips on how to proceed or what steps I may be missing? I'm completely new to Qlik and trying to test out connecting to ArcGIS Online to create reports.

[Nabeel_Asif]

Hi Mitchell, if the data load in Qlik finishes really quickly without an error message it might mean the output from the URL isn't what you expect. You can try pasting the URL you used in the GeoAnalytics connector in your browser to see the output. You should also check the Data Model Viewer in Qlik Sense to see a preview of the data that has been loaded into your app.

[wallacechen]

Hi Nabeel  Is it possible geoanalytics can integrate with Arcgis default map service without using wms wfs tms formats ?  since this example is for polygens , points and lines using shape files via Json, it is possible also to consume drone/uav images that  is put on top of earth basemap from arcgis to geoanalytics ?  thanks

[Nabeel_Asif]

Hi Wallace, for a custom background service you can use the GeoData Layer‌ in the frontend. This layer accepts GeoJSON files as well as WFS, WMS and TMS.

I haven't looked into using images or raster data with GeoAnalytics, but think this would be done through a WMS or TMS service. You may get better help by posting a question on the Qlik Community forum.

[Anonymous]

Hi Nabeel, i loaded the script into qlik but i didn't find the token. could you tell me where i can find the token? thanks

[Nabeel_Asif]

Hi Peter, if the token was generated successfully you will get a 'access_token' in your Qlik data model. You can just add the field to a table to view it. If you don't see the field, it means something went wrong when requesting the token from ArcGIS using the REST connection.

[Anonymous]

hi Nabeel, thanks for your reply. I didn't see the token when i put the token into the table so i think there might be some problems with REST. I put the same staff into REST but i am not sure the script i made.after i loaded the connection, i should put:LET vToken = Peek('access_token', 0, 'root'); in the script right? then i just continue with the code shown in the picture?

should i put the exactly same code or is there any place i should change? thanks!

[Nabeel_Asif]

You'd just need to change the 'LIB CONNECT TO' bit to the name of your data connection.

Other than that, an issue with this script is probably because of the REST connector settings.

[Matsa]

Thank you @Nabeel_Asif  for the explanation.
Is it same in qlik view as well?
because I follow the steps in qlikview 11 and it gave me an error 'Failed to load'

I created Json from query on  arcgis map server.
and then here is my setting :

Thank you in advance.
Regards,
Matsa

[Nabeel_Asif]

Hi @Matsa , yes the connector settings would be the same with QlikView. However, I see that you are using the Simplify operation with Point geometries, which doesn't make sense as only polygon or line layers can be simplified. Try using the Load operation instead.

If you still have issues you can check the JSON output of the Esri query in a browser.