Qlik engine memory usage

[Isaacdvory]

Hello,

I'm fairly new to qlik, I'm working on an existing app (if that how it's called) that someone else built and is running on AWS r5.2xlarge .

Recently our users started complaining that they can't reach the application and it's "stuck", when connecting with RDP to the server I notice the memory usage is high, 60%+ and also the CPU . sometimes it 90%+ and it's hard to even connect to the server.

Initially we figured maybe a task is taking a long time, but I checked and there was no task running... we did had some tasks taking a long time and failing but I optimized the code and they are OK now.

So my question is, how do I debug this memory issue when it happens? is there a log I can read and see what is running / happening at the time it consumes memory? 
I tried asking the users if there is anything special they did but they claim they use it normally, same things they did the day before when it didn't happen...

When I debugged the tasks I checked the logs at c:\programdata\qliq\sense\log\script and it was fairly easy to read and I could clearly see when SQL command was taking a long time. I'm hoping there is something similar I can see for this memory issue. 

I would also love to know a bit more about how to test the issue proactively but I don't know how it works exactly - so for example, when I go into an app, it has multiple sheets under it, is it a good test to just to into each one and see if the memory spikes or it also depends on the search I make in each one?  so if I go into one and make a search for data of 2 days or if I make a search for 180 days, it will have a different impact? or it doesn't matter and once I make initial search qlik fetch all the data into memory?

Thanks.

[Øystein_Kolsrud]

More details on how the engine memory behaves in your case would be necessary to understand if this is a problem or not, but during normal operations the engine process will after a while consume memory up to a limit know as "Working set low". There is a really good white paper on the topic here:

https://community.qlik.com/t5/Official-Support-Articles/Qlik-Engine-Memory-Management/ta-p/1710559

Check out the pdf-file in that page. The bottom line is that the engine has a quite elaborate memory allocation flow, and the process consumes a lot of memory is not typically a problem. Most of that memory is consumed by evaluation cache entries and the engine will purge entries from this cache when necessary in order to try to keep memory within the "Working set low" limit. The exact level is configurable, but default setting is 70% of memory. There is also another upper limit known as "Working set high", and if this level is reached, then the engine goes into a more restrictive memory management flow to void a crash due to out-of-memory condition. Information on how to configure those settings can be found here:

https://community.qlik.com/t5/Official-Support-Articles/Qlik-Sense-Engine-Enterprise-on-Windows-How-...

Hope that helps!