topic Re: Qlik Sense on Windows (May 2022) - API - How to get apps list and data load script in Integration, Extension & APIs

Qlik Sense on Windows - API - How to get apps list and data load script?

LDR — Thu, 11 May 2023 14:45:16 GMT

Hi Qlikers,

First at all I have to apologyze if I'm asking something too easy, at this moment it isn't for me. I'm trying to learn from zero how to use QS APIs and everything is new for me.

My goal is to create an external script in order to obtain a list of apps from my QS environment and for each one, I want to get the content that you can see via Data Load Editor, the reason is because I want to know the different data sources that any app is using (QVDs, Excel files, tables from any database, etc.).

I was able to do something similar developing a Qlik app but now I want to obtain this data via API. My first tests were using PowerShell. Let me show you my example:

$hdrs = @{}
$hdrs.Add("X-Qlik-xrfkey","12345678qwertyui")
$hdrs.Add("X-Qlik-User","UserDirectory=DOMAIN; UserId=USER1")
$cert = Get-ChildItem -Path "Cert:\CurrentUser\My" | Where {$_.Subject -like '*QlikClient*'}
$url = "https://QSHOST:4242/qrs/about?xrfkey=12345678qwertyui"
Invoke-RestMethod -Uri $url -Method Get -Headers $hdrs -Certificate $cert

It worked for me returning the output below:

The next step is to obtain the list of apps so I think that I need to change the method-type to POST, I mean something like that:

$body = '{
"UserDirectory": "DOMAIN",
"UserId": "USER1",
"Attributes": [
{
"name": "GETAPPLIST",
"handle": "-1",
"method": "GetDocList",
"params": []
}
]
}'
$hdrs = @{}
$hdrs.Add("X-Qlik-xrfkey","12345678qwertyui")
$hdrs.Add("X-Qlik-User","UserDirectory=DOMAIN;UserId=USER1")
$cert = Get-ChildItem -Path "Cert:\CurrentUser\My" | Where {$_.Subject -like '*QlikClient*'}
$url = "https://QSHOST:4242/qrs/about?xrfkey=12345678qwertyui"
Invoke-RestMethod -Uri $url -Method Post -Body $body -ContentType 'application/json' -Headers $hdrs -Certificate $cert

In this case the result I'm obtaining is the next below:

So, my questions are:

- Is it possible to do what I want via PowerShell? Can you show me any example? else what developing language you recommend me?

Thanks for your time 😉

Re: Qlik Sense on Windows (May 2022) - API - How to get apps list and data load script

Øystein_Kolsrud — Fri, 21 Oct 2022 07:24:32 GMT

The GetDocList endpoint is part of the engine API which is not a REST API. The engine API is instead based on JSON RPC over websockets to make it more interactive, which means you will have to connect a websocket to the engine and send your messages there.

I don't have too much experience with doing this in PowerShell (as I typically turn to C# when doing stuff like this), but it's certainly possible to do so. This thread should give you some pointers as to how to proceed:

https://community.qlik.com/t5/Integration-Extension-APIs/edit-script-load-editor-with-net-SDK/td-p/1755324

Re: Qlik Sense on Windows (May 2022) - API - How to get apps list and data load script

rzenere_avvale — Fri, 21 Oct 2022 07:27:21 GMT

Hey there @LDR ,

To obtain such information via Powershell, I would recommend Qlik-Cli-Windows
It already has many pre-configured functions, like the one to extract the list of applications in your environment

For the script... As far as I know, there is no API that you can call from Powershell to obtain it; probably you would need to take a look at Engine APIs
Otherwise, through Powershell you should be able to require the logs of an application

I hope this helps,
Riccardo

Re: Qlik Sense on Windows (May 2022) - API - How to get apps list and data load script

LDR — Fri, 21 Oct 2022 11:44:47 GMT

Hi Oystein,

If you'd have to query the app list we have and get the script for each app what would be your approach? I mean, would you use C#? According your answer, can you tell me how is configured your developing environment?

I'm asking you that becase I want to use the right tools for obtaining the data I need.

Best regards

Re: Qlik Sense on Windows (May 2022) - API - How to get apps list and data load script

LDR — Fri, 21 Oct 2022 11:46:38 GMT

Hi @rzenere_avvale

I have to ask you the same than to @Øystein_Kolsrud . If you would have to obtain the data I want how would you do it?

Please, forget Powershell option. If I have to use C#, Java, Python, etc. I'll do it but I want to learn how it has to be done.

Thanks

Re: Qlik Sense on Windows (May 2022) - API - How to get apps list and data load script

Øystein_Kolsrud — Fri, 21 Oct 2022 12:21:05 GMT

I'm not sure I understand exactly what end result you want, but if you want to access the scripts of all apps, then you could use an approach like the one below. It relies on the Qlik Sense .NET SDK for engine API access and QlikSenseRestClient for REST access.

// Get list of app IDs through REST endpoint. var client = new RestClient(url); client.AsApiKeyViaQcs(key); var appItems = client.Get<JObject>($"/api/v1/items?resourceType=app")["data"].OfType<JObject>().ToArray(); var appIds = appItems.Select(item => item["resourceId"].Value<string>()); // Get scripts for apps. var location = QcsLocation.FromUri(url); location.AsApiKey(key); foreach (var appId in appIds) { using var app = location.App(appId, noData: true); var script = app.GetScript(); Console.WriteLine($"{appId} script length: {script.Length}"); }

The two libraries are both available from NuGet:

https://www.nuget.org/packages/QlikSense.NetSDK/
https://www.nuget.org/packages/QlikSenseRestClient/

But if you are looking for ways to list data sources for apps, then you might want to have a look at the lineage endpoint to see if that gives you the information you need:

https://qlik.dev/apis/rest/apps/#%23%2Fentries%2Fv1%2Fapps%2F-appId%2Fdata%2Flineage-get

Re: Qlik Sense on Windows (May 2022) - API - How to get apps list and data load script

Øystein_Kolsrud — Fri, 21 Oct 2022 13:57:44 GMT

Oh, sorry... Just realized the question concerned Client Managed, not QCS. In that case you could do something like this:

And I think you should be able to get lineage data from the engine. I haven't really experimented with that feature yet tough. But you could try something like this:

var lineage = app.GetLineage(); Console.WriteLine(string.Join(Environment.NewLine, lineage.Select(l => l.Discriminator)));

Re: Qlik Sense on Windows (May 2022) - API - How to get apps list and data load script

jcartermd — Mon, 24 Oct 2022 05:28:09 GMT

Thank you @Øystein_Kolsrud that was very helpful. I'm also starting with almost 0 knowledge of APIs and C# and had an identical request for QSE on Windows.

I combined the above loop example with Qlik Sense: Getting started with the .NET SDK - Qlik Community - 1716231 and Qlik Sense: How to export app scripts - Qlik Community - 1717436 (and a little Stack Overflow) to end up with the C# code below, which worked out.

using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using System.IO; using Qlik; using Qlik.Engine; namespace ConsoleAppQlik { internal class Program { static void Main(string[] args) { FileStream ostrm; StreamWriter writer; TextWriter oldOut = Console.Out; Uri uri = new Uri("https://qlikserver3.domain.local/"); //change to your server URL ILocation location = Qlik.Engine.Location.FromUri(uri); //location.AsNtlmUserViaProxy(proxyUsesSsl: true); using (var hub = location.Hub()) { var appIds = location.GetAppIdentifiers(); foreach (var appId in appIds) { var app = location.App(appId, noData: true); var script = app.GetScript(); Console.WriteLine($"{appId.AppName} - {appId.AppId} script length: {script.Length}"); try { //places files in C:\Users\Your_Name\source\repos\ConsoleAppQlik\ConsoleAppQlik\bin\Debug ostrm = new FileStream($"./{DateTime.Now.ToString("yyyy-dd-M--HH-mm")} - {appId.AppName}.txt", FileMode.OpenOrCreate, FileAccess.Write); writer = new StreamWriter(ostrm); } catch (Exception e) { Console.WriteLine("Cannot open Redirect.txt for writing"); Console.WriteLine(e.Message); return; } Console.SetOut(writer); Console.WriteLine(script); Console.SetOut(oldOut); writer.Close(); ostrm.Close(); Console.WriteLine($"{appId.AppName} script export complete."); }; Console.WriteLine("Press enter to close..."); Console.ReadLine(); } } } }

Re: Qlik Sense on Windows (May 2022) - API - How to get apps list and data load script

LDR — Tue, 25 Oct 2022 11:17:30 GMT

HI @Øystein_Kolsrud and @rzenere_avvale

Thanks a lot for your examples. I couldn't test them yet because my environment (MS Visual Stucio) is claiming when I try to execute. That's something that I have to solve firtst.

On the other hand, I'd like to ask you if you know how to do the same with Python or Java without using QLIKSense.NetSDK package. Else, don't worry, your help was great.

Thanks

Re: Qlik Sense on Windows (May 2022) - API - How to get apps list and data load script

Øystein_Kolsrud — Tue, 25 Oct 2022 11:28:41 GMT

There are no official libraries available for Java, but for JavaScript you can use Enigma:

https://help.qlik.com/en-US/sense-developer/August2022/Subsystems/APIs/Content/Sense_ClientAPIs/enigmajs/enigmajs-introduction.htm

There's also the platform-sdk where you have Python support, but that's primarily intended for QCS. I don't know if it's possible to also use it for client managed:

https://qlik.dev/libraries-and-tools/platform-sdk

Re: Qlik Sense on Windows (May 2022) - API - How to get apps list and data load script

LDR — Wed, 26 Oct 2022 13:09:24 GMT

Hi @Øystein_Kolsrud

I'm thinking that my fail was to think that any API service for Qlik could be queried by any solution and so far I have understood:

For topics related to QMC we can use QRS REST API. With this option we can't open an app and extract its script. With this option we can use tools like curl, Postman, etc. because websockets aren't necessary. Saying it in a different way: with QMC I can obtain a list of apps but I can't open each one and see its objects, etc.
For topics related to apps we can use QPS API. Then we can open an app and extract its script, lineage, etc.. Whit this option we can't use curl, Postman, ... because we need to be able to create websockets. Then JS, .NET, ... are the options we have to use.

Please, can you confirm me if I'm right.

Thanks for your support

Re: Qlik Sense on Windows (May 2022) - API - How to get apps list and data load script

Øystein_Kolsrud — Wed, 26 Oct 2022 13:45:37 GMT

I'd say your first bullet is correct, but the second one is rather inaccurate. First of all, QPS typically refers to the Qlik Proxy Service (QRS refers to Qlik Repository Service). Both QRS and QPS APIs are based on REST:

Websockets on the other hand, are required to interact with the Engine API:

Engine API

The use of websockets is not really tied to a particular technology, and in theory you should be able to use it from any language you want, but this API is somewhat more complicated than simply using REST, so having a library for interacting with it that takes care of the low level stuff highly simplifies things.

Mostly when working with APIs for the client managed version of Qlik Sense you will be interacting with the Repository API and/or the Engine API. The Repository API focuses on App environment like streams, owners, publish status etc, while the Engine API focuses on app contents like script, visualizations and bookmarks.

There is some overlap between the two (you can for instance change the title of an app from the QMC), but in general that is the distinction between the two.

Re: Qlik Sense on Windows (May 2022) - API - How to get apps list and data load script

LDR — Wed, 26 Oct 2022 16:32:16 GMT

Hi @Øystein_Kolsrud

Ok, thanks for making it more clear. So the summary could be if I want to work with the app content (script, lineage, ...) I have to play with the Engine API.

Knowing that I'm going to improve my knowledge of JS and to try to play with Engine API and Enigma.js.

Thanks a lot

Re: Qlik Sense on Windows (May 2022) - API - How to get apps list and data load script

NadiaB — Tue, 01 Nov 2022 15:59:09 GMT

Hi @LDR

As @Øystein_Kolsrud Engine API is the way to go, good thing is that code in .net C# can be used in PowerShell if you try to use/convert code from c#.

Just wanted to provide the URL for QRS API as I noticed the method you are attempting to use doesn't have a body, probably you were looking for the app/full endpoint https://help.qlik.com/en-US/sense-developer/August2022/APIs/RepositoryServiceAPI/index.html

The documentation to our Engine API is found here:

https://help.qlik.com/en-US/sense-developer/May2022/Subsystems/EngineAPI/Content/Sense_EngineAPI/introducing-engine-API.htm

Hope it helps!