QlikSense connection to ADLS

[vaibhav83gupta]

Is it possible to connect directly to Azure data lake storage(ADLS) from Qlik? i know powerBI provides a default connector for ADLS.

I read connecting via spark on HDInsights, but does that work?

Thanks,

Vaibhav

[CarmenReilly]

Hi Vaibhav,

Could you do that via REST API?

ADLS: Azure Data Lake Store REST API | Microsoft Docs

Qlik Sense: https://help.qlik.com/en-US/connectors/Subsystems/REST_connector_help/Content/REST-connector.htm

-Carmen

[dsmithqlik]

Gupta,   Were you ever able to make this work?   I am working on using the Rest connector but could use some guidance if someone already has done this. 

[vaibhav83gupta]

Hi David,

No, i have not been able to. I am facing issues with Authentication. usually authentication tokens are created for short time in ADLS. these tokens need to be refreshed dynamically.

The authentication needs to be passed in query headers in Rest API connector.

Regards

vaibhav

[vaibhav83gupta]

Ok. i have been able to connect to ADLS and read data in Qliksense.

i have done it using authorization code generated by firing a POST statement to generate a access token.

The token is eligble only for 2 hours before it expires.

To create a permanent solution -

1. We need to look at reading data from ADLS using login credentials and not using access token. - not sure if adls allows it.

2. somehow create qliksense connection using a file which can be imported before reload starts. The token can be replaced in the file by a batch script or any other program.

[dsmithqlik]

Vaibhav,

Are you able to provide any details on how you connected?  I assume the Sense REST connector or are you writing a custom connector?

[vaibhav83gupta]

Hi, I used the Qlik Rest API connector.

Following are the steps -

1. Rest API needs the following to be configured –
   1. URL – this is the URL for the data file uploaded in ADLS. This is can be fetched from properties of the file in ADLS.  Syntax - https://<DLSname>.azuredatalakestore.net/webhdfs/v1/<folderpath>/<filename.ext>?op=OPEN&read=true
   2. Query headers – this is used to pass the authorization codes to ADLS. Authorization codes consists of a “Access Token” which is generated by ADLS using a POST method. The access token expires in 30mins which can then be refreshed to increase validity to 2 hours. Syntax - Authorization: Bearer <Token>

2. To generate the Access Token, the post statement has to be executed with parameters as shown below  -

POST https://login.microsoftonline.com/<Tenant-id>/oauth2/token

[

{"key":"grant_type","value":"authorization_code","description":""},

{"key":"client_id","value":"<client-id>","description":""},

{"key":"code","value":"<code>","description":""},

{"key":"resource","value":"https://datalake.azure.net/","description":""},

  {"key":"client_secret","value":"<client-secret>","description":""}

]

*Client id, Code , Client-secret needs to be fetch from Azure account.

The post statement returns the Access Code and Refresh Code, which are then used in Rest API connection.

Regards
Vaibhav

[DavidFreriks]

Hi Viabhav -- I'm stuck on the last step trying to get my auth token...

I've gotten my client_id, secret key, and code (which is a process into itself!).

But get this error when I construct the post statement to get my oauth token:

Any ideas?

Here's my POST syntax (full values redacted, but valid):

{"key":"grant_type","value":"authorization_code","description":""},

{"key":"client_id","value":"8140dd8c-......","description":""},

{"key":"code","value":"AQABAA......","description":""},

{"key":"resource","value":"https://datalake.azure.net/","description":""},

{"key":"client_secret","value":"3j/0I8h.....","description":""}

Thanks!

[vaibhav83gupta]

Hi David, Sorry i was out for sometime.. Is this resolved?

[suheshreddy]

Hi Vibhav,

I got stuck at post request, Can you please let me know the process to get code from Azure.

I have registered the app and got client id and secret.

Thanks for your help.