Qlik Community

Qlik Sense Integration, Extensions, & APIs

Discussion board where members can learn more about Integration, Extensions and API’s for Qlik Sense.

Not applicable

First steps in Qlik Sense .NET SDK saga. What I did Wrong?

One nice morning I got a task to research a way to browse Qlik Sense apps and embed sheets from it to our app. I looked to site dev center and "wow! there is a ready .net sdk for me, it should be simple enough to use". Naive, naive me...

Ok, happy me created a new project to test that SDK and made little proof-of-concept app. Found package in nuget and what did I see?

First anxious bell ranged:

p1.pngSo, it has strong restrictions for using new Newtonsoft.Json? Why? How to use it in modern apps? Downgrade Newtonsoft.Json or just force SDK to work with newer version (hey, it is beta 9 on the way!) and hope for the best? Anyhow, let's go further.

As I see in documentation it should be pretty easy to connect. Get a sample:

var location = Location.FromUri(new Uri("https://sense-demo.qlik.com:4848"));

using (IHub hub = location.Hub())

{

    Console.WriteLine(hub.ProductVersion());

}

And it fails with exception, friendly as hell:

'Qlik.Engine.Communication.CommunicationErrorException' occurred in Qlik.Engine.dll

Additional information: Connection failed after 4 attempts. Unable to keep connection open: Failed to ensure open connection: One or more errors occurred.

Ok, start playing with url, trying remove port, trying other port, trying wss protocol. Ouch, wss protocol gives another exception. not so friendly though:

An unhandled exception of type 'System.TimeoutException' occurred in Qlik.Engine.dll

Additional information: Method "QTProduct" timed out

How? Why? What the hell?

Few headaches and solution found. I just need to use async versions like that:

var location = Location.FromUri(new Uri("wss://sense-demo.qlik.com"));

using (IHub hub = await location.HubAsync())

{

    Console.WriteLine(await hub.ProductVersionAsync());

}

It works. No idea why. I feel like shaman. But why that call takes whole 30 seconds?

Ok, lets try to actually browse something:

var location = Location.FromUri(new Uri("wss://sense-demo.qlik.com"));  

var appIdentifiers = await location.GetAppIdentifiersAsync();

foreach (var appIdentifier in appIdentifiers)

{

        Console.WriteLine($"appIdentifier Id = {appIdentifier.AppId} name = {appIdentifier.AppName}");

        using (var app = await location.AppAsync(appIdentifier))

        {

              var sheets = await app.GetSheetListAsync();

              foreach (var sheet in sheets.Items)

              {

                  Console.WriteLine($"sheet id = {sheet.Info.Id} name = {sheet.Data.Title}");

              }

        }

}

Hello exception, my old friend:

Method \"QTProduct\" timed out

Why?? It worked just a minutes ago! Ok, what changed? I removed lines about hub!

Ok, lets try that:

var location = Location.FromUri(new Uri("wss://sense-demo.qlik.com"));

using (IHub hub = await location.HubAsync()) {}

var appIdentifiers = await location.GetAppIdentifiersAsync();

foreach (var appIdentifier in appIdentifiers)

{

      Console.WriteLine($"appIdentifier Id = {appIdentifier.AppId} name = {appIdentifier.AppName}");

      using (var app = await location.AppAsync(appIdentifier))

      {

            var sheets = await app.GetSheetListAsync();

            foreach (var sheet in sheets.Items)

            {

                  Console.WriteLine($"sheet id = {sheet.Info.Id} name = {sheet.Data.Title}");

            }

      }

}

I inserted line two. It starts working. WAT? You know, it was a video back then about strange things in ruby and javascript - WAT. I felt the same... it is like going through a minefield, step aside and boom, exception thrown.

But wait, let's measure execution time of that small script. 3 minutes 17 seconds. Wow. Cache, my friend, here I come...

Nevertheless, it works. Let's try to put in a real application. it is easy, you just need to fight with package old dependencies. right? Ha-ha, tells SDK! Project stops compile. With very funny errors:

The call is ambiguous between the following methods or properties:

'System.Linq.Dynamic.DynamicQueryable.OrderBy<T>(System.Linq.IQueryable<T>, string, params object[])' and

'System.Linq.Dynamic.DynamicQueryable.OrderBy<T>(System.Linq.IQueryable<T>, string, params object[])'

Nice! I just started to feel boring, thanks SDK. Dust off my old reflector and let's cut SDK open.

And in a project Qlik.Sense.ExtendedFramework I saw beautiful:

p2.pngReally? They put in in a library source just like that? Qlik, please, please beat developer who did it with a most heavy book of Martin Fowler you can find.

And that is almost all fun I got. But I still have some questions like why the hell very useful method that are working perfectly fine in your plugin for visual studio are hidden from other developers. Like OpenAsNtlmUserViaProxy which accepts networkcredentials and lots of others (thank you, reflector, for many knowledge).

Thank you, Qlik .NET SDK for such user-friendly, clean and easy SDK!

1 Solution

Accepted Solutions
Employee
Employee

Re: First steps in Qlik Sense .NET SDK saga. What I did Wrong?

That's a lot of issues you ran into in one go. There is logic to it though:

1) The .Net SDK has a dependency restriction on Newtonsoft for version 7.0.1 due to the way schema management was broken out in Newtonsoft after that version. Going to newer versions would break backwards compatibility, which is why have have this conservative approach.

2) Port 4848 is typically the port used when connecting directly to the engine of a desktop installation. 443 is the standard port to use for the proxy of server installations, and that is also the port to use when connecting to sense-demo.qlik.com, however, as you yourself noted, just omitting the port is usually the way to go.

3) The 30-second delay you are seeing is due to how anonymous mode is handled (or not handled) in the .Net SDK. What happens is that the connection will try to authenticate, but fail after a timeout of 30 seconds. As no authentication is actually required for anonymous mode, it will just continue and start sending the requests. Support for anonymous mode will be available in version 3.0 that I believe was released today. With that version you should configure the location by writing "location.AsAnonymousUserViaProxy()". However in your case you could simply set the timeout to something low for the first hub connection like this:


var tmpTimeout = QlikConnection.Timeout;

QlikConnection.Timeout = 500; // Timout of 500 ms

var location = Qlik.Engine.Location.FromUri(new Uri("https://sense-demo.qlik.com"));

using (IHub hub =  await location.HubAsync())

{

    QlikConnection.Timeout = tmpTimeout; // Reset to original timeout

    Console.WriteLine(await hub.ProductVersionAsync());

Notice that it is the call to HubAsync (which sets up the initial websocket connect) that takes time. Ones it has gotten through that timeout, the "ProductVersionAsync" call should be fast.

4) Your friend "QtProduct" is used as a ping method towards the engine. It's the first message sent to ensure that a valid communication channel is open. A timeout on that method when opening a connection basically means that you are not able to talk to the engine.

5) The timeout you get from "GetAppIdentifiersAsync" is interesting! I see why it happens though, but that is something we will fix. That method exhibits some behavior that is not desirable from an async method. What happens is that when you add the "await location.HubAsync()" is that the websocket configuration (and therefore authentication timeout) will be consumed by that call. As you call the "HubAsync" directly, it will not suffer from a time out. On the other hand, "GetAppIdentifiersAsync" will open the websocket in a way that will trigger a timeout.

6) Dependencies on the "ExtendedFramework" dll has been removed in .Net SDK version 3.0, so you should no longer experience that problem if you go to that version.

9 Replies
Employee
Employee

Re: First steps in Qlik Sense .NET SDK saga. What I did Wrong?

First, I'm sorry for laughing at your pain but this was incredibly hilarious! Kudos!

Secondly, the reason you are seeing very slow response times when hitting sense-demo.qlik.com is a long story but it's related to the way that specific server(s) is set up. Running the SDK against any other instances of Qlik Sense and you should see considerably faster response times.

Not applicable

Re: First steps in Qlik Sense .NET SDK saga. What I did Wrong?

Thank you, I'll check it when I'll be able to connect to our test instance of Qlik Sence. For now I am on stage where I see exceptions trying to connect, but I have few ideas left about what to check in its settings.

But if you can give me a tip about what may be wrong - I'd really appreciate it. For now I allowed anonymous access (it became accessible from browser without authorization), and I opened all required ports I could find in proxy settings. But still no luck connecting from SDK, even with code that works for https://playground.qlik.com and https://sense-demo.qlik.com

Employee
Employee

Re: First steps in Qlik Sense .NET SDK saga. What I did Wrong?

In QMC under the virtual proxies we have something we call the Websocket whitelist.

This is a list of allowed hosts that are allowed to establish a websocket connection, for playground and sense-demo we have localhost listed as a valid origin.

Make sure that whatever http origin your application is using is present in that whitelist otherwise we will terminate your incoming websocket request.

Not applicable

Re: First steps in Qlik Sense .NET SDK saga. What I did Wrong?

I've added 127.0.0.1 and localhost to that list. I've also tried to enter there my external IP address. No luck =(

For now in that list: 127.0.0.1, localhost, Azure VM external IP, Azure VM url, Azure VM inner name.

The most worrying thing that I can not even connect from that very same VM to it, and the fact that I am receiving communication error, not "Authorization failed" or smth more meaningful.

konrad_mattheis
Contributor III

Re: First steps in Qlik Sense .NET SDK saga. What I did Wrong?

Hi Antonio,

I read your article with a wide grin. Yes the SDK is not always like an Developer expect, but together with my old friends reflector and reflection I could work around all these obstacles.

See it as a entrance check. Only well experience .NET Developer get it really working :-)


bye Konrad

Employee
Employee

Re: First steps in Qlik Sense .NET SDK saga. What I did Wrong?

Both the playground and sense-demo servers are running with anonymous users so that could also be a difference.

Check your virtual proxy in QMC under the Authentication tab to see if you have enabled anonymous access. The .NET SDK also gives you some utility functions to connect as a NTLM user if you would prefer that over anonymous access.

Also, the above mentioned deployments have a valid SSL cert installed. The certificates that are generated during the install of Qlik Sense are by design self-signed and thus sometimes not trusted.

Try adding

ServicePointManager.ServerCertificateValidationCallback = delegate { return true; };

before any connection to Qlik Sense is made to automatically trust a self-signed cert.

Employee
Employee

Re: First steps in Qlik Sense .NET SDK saga. What I did Wrong?

That's a lot of issues you ran into in one go. There is logic to it though:

1) The .Net SDK has a dependency restriction on Newtonsoft for version 7.0.1 due to the way schema management was broken out in Newtonsoft after that version. Going to newer versions would break backwards compatibility, which is why have have this conservative approach.

2) Port 4848 is typically the port used when connecting directly to the engine of a desktop installation. 443 is the standard port to use for the proxy of server installations, and that is also the port to use when connecting to sense-demo.qlik.com, however, as you yourself noted, just omitting the port is usually the way to go.

3) The 30-second delay you are seeing is due to how anonymous mode is handled (or not handled) in the .Net SDK. What happens is that the connection will try to authenticate, but fail after a timeout of 30 seconds. As no authentication is actually required for anonymous mode, it will just continue and start sending the requests. Support for anonymous mode will be available in version 3.0 that I believe was released today. With that version you should configure the location by writing "location.AsAnonymousUserViaProxy()". However in your case you could simply set the timeout to something low for the first hub connection like this:


var tmpTimeout = QlikConnection.Timeout;

QlikConnection.Timeout = 500; // Timout of 500 ms

var location = Qlik.Engine.Location.FromUri(new Uri("https://sense-demo.qlik.com"));

using (IHub hub =  await location.HubAsync())

{

    QlikConnection.Timeout = tmpTimeout; // Reset to original timeout

    Console.WriteLine(await hub.ProductVersionAsync());

Notice that it is the call to HubAsync (which sets up the initial websocket connect) that takes time. Ones it has gotten through that timeout, the "ProductVersionAsync" call should be fast.

4) Your friend "QtProduct" is used as a ping method towards the engine. It's the first message sent to ensure that a valid communication channel is open. A timeout on that method when opening a connection basically means that you are not able to talk to the engine.

5) The timeout you get from "GetAppIdentifiersAsync" is interesting! I see why it happens though, but that is something we will fix. That method exhibits some behavior that is not desirable from an async method. What happens is that when you add the "await location.HubAsync()" is that the websocket configuration (and therefore authentication timeout) will be consumed by that call. As you call the "HubAsync" directly, it will not suffer from a time out. On the other hand, "GetAppIdentifiersAsync" will open the websocket in a way that will trigger a timeout.

6) Dependencies on the "ExtendedFramework" dll has been removed in .Net SDK version 3.0, so you should no longer experience that problem if you go to that version.

Not applicable

Re: First steps in Qlik Sense .NET SDK saga. What I did Wrong?

Yes, that was enabled. But I migrated to version 3 SDK as Kolsrud subjected and it started to work. Thank you!

Not applicable

Re: First steps in Qlik Sense .NET SDK saga. What I did Wrong?

I've tried .NET SDK 3 and it seems to be connecting to my instance successfully with small change:

location.IsVersionCheckActive = false;

Thank you!

Community Browser