topic Looking to extract all data from a data source using backend (non-javascript) API in App Development

Looking to extract all data from a data source using backend (non-javascript) API

clondon — Fri, 28 May 2021 20:22:08 GMT

Cinch is a marketing automated start up and we have a client that uses Qlik Sense as a data warehouse and visualization tool. I would like in an automated schedule (e.g., once a day) to download the data from a data source. I've been given an API key but I don't seem to see any way to extract the data.

Re: Looking to extract all data from a data source using backend (non-javascript) API

rwunderlich — Fri, 28 May 2021 21:54:28 GMT

Just to clarify a few things.

1. You mentioned API key. Is this customer using Qlik Sense in Qlik Cloud or on-premise at customer?

2. Note that Qlik Sense loads data from data sources -- database tables, excel files, etc -- and stores them in a Qlik Application data model for use by visualizations in this chart. Do you want to:
a) Extract the data from the Application data model?
b) Extract/copy the input data sources that Qlik used to build the application data model?

It sounds like "a" but thought I would confirm before going further.

-Rob

Re: Looking to extract all data from a data source using backend (non-javascript) API

clondon — Fri, 28 May 2021 22:14:51 GMT

Thanks, you are correct. I have an API key for Qlik Sense and I want to try and do "a", Extract the data from the Application data model.

Re: Looking to extract all data from a data source using backend (non-javascript) API

rwunderlich — Sat, 29 May 2021 14:28:41 GMT

The simplest way to reuse that data is for your customer to store the data in CSV files during the ETL process and have you pull the csv files. In the Qlik Script, which is the language they use to ETL and build the data model, they can add the statement(s).

Store qliktable into lib://datafiles/myqliktable.csv (txt);

Then download the csv files. If you don't want to take that route but instead want to extract directly from the app model yourself, you can use the JSON-RPC API (Engine API). Use the GetTableData() method to pull data directly from a table or create a hypercube that defines the specific joined fields you want.

There are several libraries that wrap the JSON-RPC API and make life easier -- enigma.js for javascript/nodejs, enigma.go for golang and .Net SDK

There is also a REST wrapper https://github.com/ralfbecher/q-risotto but I'm not sure the status of this project, check with @rbecher if you are interested in this one.

-Rob
http://masterssummit.com
http://qlikviewcookbook.com
http://www.easyqlik.com

Re: Looking to extract all data from a data source using backend (non-javascript) API

clondon — Mon, 07 Jun 2021 22:45:15 GMT

Is it possible to download the files generated by the Qlik Script via an API?

Re: Looking to extract all data from a data source using backend (non-javascript) API

rwunderlich — Fri, 11 Jun 2021 14:47:56 GMT

If by "files generated", you mean the aforementioned csv files, yes. Use an FTP or other file transfer API. These are just just standard files.

-Rob