article Qlik Sense Engine Enterprise on Windows: How the memory hard max limit works in Official Support Articles

Qlik Sense Engine Enterprise on Windows: How the memory hard max limit works

Bastien_Laugiero — Thu, 05 Dec 2024 14:03:13 GMT

The Qlik Sense Engine allows for a Hard Max Limit to be set on memory consumption.

To be able to enforce "Hard max limit" the engine requires access to a specific endpoint in the Windows operating system API.

If the operating system blocks calls to this endpoint, then an attempt to use the configuration "Hard max limit" will fail, and the engine will instead operate according to the "Soft max limit" configuration.

Refer to the documentation provided by Microsoft fur forther information: "SetProcessWorkingSetSizeEx"

How to review your memory settings in Qlik Sense or configure the Hard Max Limit:

Open the Qlik Sense Management Console in a supported browser
Navigate to Engines in the Management Console and select your engine
Expand the Advanced settings from the menu to the right
Review your memory settings and verify that Max Max Limit is selected

See Editing an engine - Qlik Sense for administrators for details.

To note:

Even with the hard limit set, it may still be possible for the host operating system to report memory spikes above the Max memory usage (%).

This is down to how the Qlik Sense Engine memory limit will be defined based on the total memory available.

Example:

A server has 10 GB of memory in total, and the Max memory usage (%) is set to 90%.
This will allow the process engine.exe to use 9 GB of memory.
Another application or process may at that point already be consuming 5 GB memory, and this will cause an overload of the system if the engine is set to use 9 GB.
The engine.exe will not be able to respect other services

The memory working setting limit is not a hard limit to set on the engine. This is a setting which determines how much we allocate and how far we are allowed to go before we start alarming on the working set beyond parameters.

Internal Investigation IDs:

QLIK-96872, HLP-17517

Re: Qlik Sense Engine - How the memory hard max limit works

QFabian — Fri, 18 Mar 2022 00:25:26 GMT

Excellent @Bastien_Laugiero , thank you

Re: Qlik Sense Engine - How the memory hard max limit works

aadil_madarveet — Mon, 21 Nov 2022 08:59:31 GMT

Thanks for the article.

Could you please explain what happens if the max memory usage is reached in a multi-node cluster. If the memory usage one node 1 is at max usage, will the resources of node 2 be used to maintain user sessions and performance. My question is basically, does Qlik manage resources spill over to next available node for the same app.

Thanks,
Aadil

Re: Qlik Sense Engine - How the memory hard max limit works

AdamBS — Wed, 17 Apr 2024 11:46:54 GMT

Does the help indicate that this Hard max limit is basically void unless the O/S adjustment is done?

https://help.qlik.com/en-US/sense-admin/February2024/Subsystems/DeployAdministerQSE/Content/Sense_DeployAdminister/QSEoW/Administer_QSEoW/Managing_QSEoW/edit-engine.htm

As per @aadil_madarveet when the limit is reached, what is the behavior of the engine? Does it just restart?

Re: Qlik Sense Engine Enterprise on Windows: How the memory hard max limit works

Sonja_Bauernfeind — Thu, 25 Apr 2024 06:46:43 GMT

Hello @AdamBS

Thank you so much for the feedback, this helps us keep our articles up to date. I've added the note regarding the Windows setting, as, yes, it is required.

As for the second question: If an app is opened at 90% usage on this node and hasn't otherwise been load balanced previously, what will happen is that the app will open while Qlik attempts to aggressively clean memory out to get back to the minimum value. This can still result in a memory overload, but does not have to.

All the best,
Sonja

Re: Qlik Sense Engine Enterprise on Windows: How the memory hard max limit works

AdamBS — Thu, 25 Apr 2024 10:55:57 GMT

Interesting. Thanks for the update, Adam

Re: Qlik Sense Engine Enterprise on Windows: How the memory hard max limit works

rwunderlich — Mon, 08 Jul 2024 15:18:27 GMT

@Bastien_Laugiero The referenced doc for configuring the OS seems to show the setting is made using code. How does a customer go about enabling this OS modification in their QS Server?

-Rob

Re: Qlik Sense Engine Enterprise on Windows: How the memory hard max limit works

VOG_JK — Fri, 22 Nov 2024 11:14:56 GMT

Hello,

as we are struggling with the same issue as @rwunderlich i wondered if there is an answer to where we can enable the O/S setting?

Is it even needed?
As far as i can tell the microsft link describes a function you call for a process e.g. the qlik engine service.
I would have guessed the settings via the qmc would call this function with the right parameter depending on what i selected in the drop down?

would be nice if someone could help me out with this one. @Bastien_Laugiero @Sonja_Bauernfeind

Many thanks, Joe

Re: Qlik Sense Engine Enterprise on Windows: How the memory hard max limit works

Sonja_Bauernfeind — Tue, 26 Nov 2024 13:53:10 GMT

Hello @VOG_JK

I'm headed to RnD to get answers for you!

All the best,
Sonja

Re: Qlik Sense Engine Enterprise on Windows: How the memory hard max limit works

Sonja_Bauernfeind — Thu, 05 Dec 2024 14:51:36 GMT

Hello @VOG_JK

I've received an answer and have updated the article accordingly. The help site will be updated with its next refresh cycle.

All the best,
Sonja

Re: Qlik Sense Engine Enterprise on Windows: How the memory hard max limit works

VOG_JK — Thu, 05 Dec 2024 15:15:20 GMT

Hey @Sonja_Bauernfeind

Thanks for the follow up!

cheers, Joe

Re: Qlik Sense Engine Enterprise on Windows: How the memory hard max limit works

afujikawa — Fri, 20 Dec 2024 06:27:47 GMT

Hello @Bastien_Laugiero and @Sonja_Bauernfeind

Is there a way to check in the Qlik Sense logs, etc., whether the operating system has blocked calls to the endpoint?

Regards,
afujikawa