Windows diagnosed a low virtual memory condition - Microsoft-Windows-Resource-Exhaustion-Detector

[jpjust]

Hi,

In one of our qliksense server, we see the following error:

Event Description:

Windows successfully diagnosed a low virtual memory condition. The following programs consumed the most virtual memory: Engine.exe (10928) consumed 11720503296 bytes, nlparser_r.exe (1020) consumed 1560207360 bytes, and Repository.exe (8108) consumed 518823936 bytes.

Any idea on where exactly to and see the cause for this low virtual memory condition?

Any experience on this issue will be appreciated.

Thanks

[Maria_Halley · Accepted solution]

@jpjust 

This is a bit diffucult to troubleshoot like this.This can be normal QlikSense behavior. 

QlikSense is an in memory application and it does consume a lot of RAM.

When you open a document on the server it gets loaded to RAM. The document seizes in RAM in not the same as on disk. It can be many times bigger in memory.

A 1 GB document could be 5-10 in RAM (or more), it depends on the data model and design of the document.

Once the document is loaded it stays in memory until the document timeout has passed (default 8h). The document timeout does not start to count down until all sessions against it is closed.

It is normal that the memory consumption stays high even if no one is logged in.

We also cache calculations and session information.  This does not get cleared when the document gets unloaded but stays in memory until the working set limit is reached.

Then we start to clear it out, starting with the data that was used the longest ago.

You can use the QIX log to see if there is any particular app that is taking up the memory

https://community.qlik.com/t5/Knowledge/How-to-enable-QlikView-QIX-performance-logging/ta-p/1716038

Here is another link with a lot of info

https://community.qlik.com/t5/Knowledge/QIX-Engine-Memory-Management-and-CPU-Utilization/ta-p/171618...

[jpjust]

Thanks Maria for the information. I have seen memory mgmt article before. I will take a look at the QIX one.

One question from your response:

"When you open a document on the server it gets loaded to RAM" - Which server do you mean? Central node or consumer node? When an user opens an app from their PC  browser, on which server RAM the app is loaded?

Thanks

[Maria_Halley]

@jpjust 

That really depends on your setup. 

From the help. (https://help.qlik.com/en-US/sense-admin/May2021/Subsystems/DeployAdministerQSE/Content/Sense_DeployA...)

When you install multiple engines and virtual proxies, you must add load balancing to the new nodes and virtual proxies. It is only on the central node that load balancing is automatically added. If you create a node without configuring the virtual proxy, the node will never actually be used. If you create a new virtual proxy, you must configure it by adding load balancing and selecting which nodes that the virtual proxy can forward work to.

The default algorithm used for load balancing is round-robin, where the load is evenly distributed between the available nodes on the multi-node site. However, any subsequent sessions from the same user/client will open on the current engine node, instead of following the round-robin.

[jpjust]

Hi Maria,

Thanks for the information.

Lets say my setup is like this, I have one central node and one consumer node.

In this case when the user opens the app from the hub, which server RAM will be used? Central or consumer node?

Thanks

[Maria_Halley]

It depends on how you have configured the virtual proxies.

See the below help section

https://help.qlik.com/en-US/sense-admin/May2021/Subsystems/DeployAdministerQSE/Content/Sense_DeployA...

[rohitk1609]

@jpjust you should let Windows handle the virtual memory. Virtual memory is not RAM. 

Pagination: Go to system, Advance settings=> Performance=> Advanced=> Change=> Allow Automatically manage paging file.

If a post helps to resolve your issue, please mark the appropriate replies as "Accept as Solution" or Like it.

Reach out to me at kumar.rohit1609@gmail.com if there is need of any clarification or assistance

Connect with me on LinkedIn https://www.linkedin.com/in/rohit-kumar-67ba152b/

To get latest updates and articles, join Qlik community at https://www.facebook.com/QlikIntellectuals

Best,

[jpjust]

Hi Rohit,

That is my setting now, please see the screen shot below.

Some times after the below error, the engine service restarts automatically.

Event Description: Windows successfully diagnosed a low virtual memory condition. The following programs consumed the most virtual memory: Engine.exe (10928) consumed 11720503296 bytes, nlparser_r.exe (1020) consumed 1560207360 bytes, and Repository.exe (8108) consumed 518823936 bytes.

How to find exactly what is causing this issue and prevent this from happening?

[Andre_Sostizzo · Accepted solution]

@jpjust, unfortunately there is not simple answer on how to optimize memory usage of your server. I'd suggest following the article below:

How to start investigating a suspected Qlik Sense Performance issue

[rohitk1609]

I have seen engine service restarts due to low virtual memory. Just look at this case, if consumption of CPU at 100%, so Windows restart that service which is spiking the CPU otherwise system will crash. I think you should analyse when this restart happens? which app is reloading that time? any aggregation or joining causing it? or drop an email to me kumar.rohit1609@gmail.com