topic Re: Iterating through the App collection makes memory usage on the server go rampant. in Integration, Extension & APIs

Iterating through the App collection makes memory usage on the server go rampant.

Tue, 12 Jul 2016 08:40:17 GMT

Hi group.

I'm trying to collect various App data to produce a Single Configurator URL using the .NET SDK. To do that, I get the whole app list, then for each app I iterate through its NxInfo objects:

IEnumerable<IAppIdentifier> appIds = hub.GetAppList();

foreach (AppIdentifier appIdentifier in appIds)

{

App app = (App)location.App(appIdentifier);

IEnumerable<NxInfo> appInfos = app.GetAllInfos();

foreach (NxInfo appInfo in appInfos)

{

// Some of the properties I'm extracting (there are others, some from App)

string appObjectType = appInfo.Type;

string appObjectId = appInfo.Id;

// Save all properties to a CSV-file - code not shown.

}

While I can extract the desired properties I get a timeout exception. What's worse, if I monitor the memory usage on the Qlik server I can see it skyrocket the moment that I start the program. I'm attaching a screenshot Task Manager during the program execution.

Is this normal behaviour? Am I starting all apps when iterating through the collection?

Re: Iterating through the App collection makes memory usage on the server go rampant.

Alexander_Thor — Tue, 12 Jul 2016 11:25:54 GMT

Yes that is expected behavior. When you open a app on the server you are loading data and the object model into memory on the server and we will allocate memory accordingly.

So in your example you are essentially opening every available doc on the server which most likely causes you to run out of RAM if it's a small box and we start swapping to disk, hence timeouts.

Luckily you can open a app without data, specify noData and you should be good

LocationExtensions.App Method

Re: Iterating through the App collection makes memory usage on the server go rampant.

Thu, 14 Jul 2016 08:30:02 GMT

Hi.

I just had the chance to try the suggested method and it works! I'm now able to iterate through the App-collection and retrieve the app metadata without getting any timeouts or a huge increase in memory consumption. Thanks!

Re: Iterating through the App collection makes memory usage on the server go rampant.

Alexander_Thor — Thu, 14 Jul 2016 09:07:08 GMT

Awesome! Also make sure that the user you are using is allocated a user license and not login licenses as that would exhaust your license pool fairly quickly.

I do believe the .Net SDK supports/will support direct connection to QIX also. That would allow you to connect using certificate authentication, bypassing the proxy and in the process not consuming any tokens. Perfect for back-end services.

Re: Iterating through the App collection makes memory usage on the server go rampant.

amien — Sat, 29 Apr 2017 10:52:13 GMT

the down side is that you won't be able to extract tables and fieldnames with nodata?

Re: Iterating through the App collection makes memory usage on the server go rampant.

Alexander_Thor — Sat, 29 Apr 2017 17:55:49 GMT

amien‌ Yes, a workaround would be to take a copy of the core application without data via something like CreateSessionAppFromApp method ‒ Qlik Sense

Then you could edit the load script of the copy to only load something like first 100 records of the tables and then reload the application. That would still be fast enough, low impact on ram and you would be able to extract metadata around tables and fieldnames within the application.