Qlik Sense Engine Enterprise on Windows: How the memory hard max limit works

[Bastien_Laugiero]

The Qlik Sense Engine allows for a Hard Max Limit to be set on memory consumption. 

To be able to enforce "Hard max limit" the engine requires access to a specific endpoint in the Windows operating system API.

If the operating system blocks calls to this endpoint, then an attempt to use the configuration "Hard max limit" will fail, and the engine will instead operate according to the "Soft max limit" configuration.

Refer to the documentation provided by Microsoft fur forther information: "SetProcessWorkingSetSizeEx"

How to review your memory settings in Qlik Sense or configure the Hard Max Limit:

1. Open the Qlik Sense Management Console in a supported browser
2. Navigate to Engines in the Management Console and select your engine
   
   
   
   
3. Expand the Advanced settings from the menu to the right
   
   
   
   
4. Review your memory settings and verify that Max Max Limit is selected
   
   

See Editing an engine - Qlik Sense for administrators for details.

To note:

Even with the hard limit set, it may still be possible for the host operating system to report memory spikes above the Max memory usage (%).

This is down to how the Qlik Sense Engine memory limit will be defined based on the total memory available.

Example:

• A server has 10 GB of memory in total, and the Max memory usage (%) is set to 90%.
• This will allow the process engine.exe to use 9 GB of memory. 
• Another application or process may at that point already be consuming 5 GB memory, and this will cause an overload of the system if the engine is set to use 9 GB. 
• The engine.exe will not be able to respect other services

The memory working setting limit is not a hard limit to set on the engine. This is a setting which determines how much we allocate and how far we are allowed to go before we start alarming on the working set beyond parameters. 

Internal Investigation IDs:

QLIK-96872, HLP-17517

[QFabian]

Excellent @Bastien_Laugiero , thank you

[aadil_madarveet]

Thanks for the article.

Could you please explain what happens if the max memory usage is reached in a multi-node cluster. If the memory usage one node 1 is at max usage, will the resources of node 2 be used to maintain user sessions and performance. My question is basically, does Qlik manage resources spill over to next available node for the same app.

Thanks,
Aadil

[AdamBS]

Hi

Does the help indicate that this Hard max limit is basically void unless the O/S adjustment is done?

https://help.qlik.com/en-US/sense-admin/February2024/Subsystems/DeployAdministerQSE/Content/Sense_De...

As per @aadil_madarveet when the limit is reached, what is the behavior of the engine?  Does it just restart?

[Sonja_Bauernfeind]

Hello @AdamBS 

Thank you so much for the feedback, this helps us keep our articles up to date. I've added the note regarding the Windows setting, as, yes, it is required. 

As for the second question: If an app is opened at 90% usage on this node and hasn't otherwise been load balanced previously, what will happen is that the app will open while Qlik attempts to aggressively clean memory out to get back to the minimum value. This can still result in a memory overload, but does not have to. 

All the best,
Sonja 

[AdamBS]

Interesting.  Thanks for the update, Adam

[rwunderlich]

@Bastien_Laugiero The referenced doc for configuring the OS seems to show the setting is made using code. How does a customer go about enabling this OS modification in their QS Server?

-Rob

[VOG_JK]

Hello, 

as we are struggling with the same issue as @rwunderlich i wondered if there is an answer to where we can enable the O/S setting? 

Is it even needed? 
As far as i can tell the microsft link describes a function you call for a process e.g. the qlik engine service. 
I would have guessed the settings via the qmc would call this function with the right parameter depending on what i selected in the drop down? 

would be nice if someone could help me out with this one. @Bastien_Laugiero @Sonja_Bauernfeind 

Many thanks, Joe 

[Sonja_Bauernfeind]

Hello @VOG_JK 

I'm headed to RnD to get answers for you!

All the best,
Sonja

[Sonja_Bauernfeind]

Hello @VOG_JK 

I've received an answer and have updated the article accordingly. The help site will be updated with its next refresh cycle.

All the best,
Sonja 

[VOG_JK]

Hey @Sonja_Bauernfeind 

Thanks for the follow up! 

cheers, Joe