In this article, we outline troubleshooting steps to identify QlikView Server Service problems related to resource issues and crashes. Examples are progressively higher RAM or CPU usage and engine crashes due to exceptions.
It also covers general guidelines on how to troubleshoot QlikView Server Service crashes.
Before we can get started, we need to look at how QlikView (or our QIX engine in general) uses resources. Material for extended reading is available here: QlikView and Qlik Sense resource usage and the Qlik Associative Engine Memory Management and CPU Uti... (including an excellent video summary that I really need to recommend).
So in order to troubleshoot resource issues and crashes, we need to first figure out if something is wrong, or if we are looking at the QlikView server having outgrown the currently available resource. Since like anything else, as usage grows, demand grows and what we currently have available just isn’t enough anymore. A bit like a plant outgrowing its pot.
But how do we identify an actual problem vs an expected behaviour?
I know, I already blamed the engine from the get-go, but we do need to make sure we aren’t pointing fingers at the wrong culprit. So, if the host operating system is running out of resources, we first want to make 100% sure that it’s the qvs.exe that’s at fault. This can be determined either by monitoring the Windows Task Manager\Processes tab directly while the problem occurs or maybe it was previously identified by a resource monitoring tool, such as Windows Performance Monitor.
If it turns out to be a qvb.exe (if the Distribution Service is on the same machine), then this got a little easier, since then it’s a reload that’s causing the problem. Troubleshooting this is, sadly, not covered in this post though. Maybe another time?
Memory usage increases gradually over time and stays stable at or around the configured Low Working Set Limit.
This is expected behaviour.
We can confirm what the Working Set is configured as in: QlikView Management Console > System > Setup > QlikView Servers > QVS@SERVER > Performance
So, it'll look a bit like:
Memory usage increases gradually over time, is stable at or around the Low Working Set Limit or at the High Limit. Users are experiencing a negative performance impact.
This may indicate that the current setup needs to be reevaluated and that more resources need to be made available, or an additional QlikView Server node needs to be added to the environment. The below steps may still be applied to find possible problem documents or objects that could be optimized.
May also look like the graph above.
Memory usage increases suddenly and leads to performance impact or QlikView Server Service crashes
Boom. Unexpected behaviour.
Often looks somewhat like this:
In this example 1 document took up the majority of available RAM, while another was loaded in after, tipping the QVS.exe to 100% memory usage.
Next, we need to start analyzing a few specific log files, and for that, we need to first identify 3 things:
The When can be identified post mortem (after the fact) by looking at when issues were reported, and hopefully by catching errors and warnings logged in the QlikView Server Event logs. But since we want to make sure we are prepared for the next time this happens, we usually recommend setting up Resource Monitoring.
Configuring resource monitoring or performance monitoring in Windows to get an overview of the situation is always recommended. See How to set up performance monitoring for QlikView Server Service(QVS) (perfmon) for a simple example.
This is where we roll our sleeves up and start digging.
The QlikView Server Service has four log files that are crucial for identifying possible issues, and I will touch briefly on all of them. For more details on logs, check out this “How to collect QlikView log files” article.
This includes engine activity. How much we will be able to read depends on log verbosity.
Includes performance information for the QlikView Server Service only. No other services or components will be logged.
VMCommitted(MB) VMAllocated(MB) VMFree(MB) VMLargestFreeBlock(MB)
I like using this one to pinpoint crashes easily, as it will show when the service starts up. And a glance at the memory statistics can already help identifying how quickly we consumed it all. Generally, we like throwing this into a QlikView or Sense App and looking at pretty graphs.
Logs user actions, such as the opening of documents, opening of sheets, bookmark selections, exports, etc.
C:/ProgramData/QlikTech/Documents/movies database.qvw SendToExcel DOMAIN\Administrator Sheet Object Document\LB04
This is what we need when we suspect user actions to be responsible for the behaviour. Like someone attempting to export a table that pulls out every last bit of data from the document, or a user-created objects with an expression that causes an exception in the engine and crashes it.
Records server-wide closed sessions. Sessions closed due to QlikView Server Service restarts should also be logged, if sessions are unaccounted for, this needs to be noted too, as it will indicate a service crash.
New! Starting from QlikView November 2018/version 12.30, you now have the possibility to capture granular usage metrics from the Qlik in-memory engine based on configurable thresholds. This provides the ability to capture CPU and RAM utilization of individual chart objects, CPU and RAM utilization of reload tasks, and more.
This log is by default not enabled so please follow the instruction provided here to enable it.
! Be very careful when enabling this, as it can generate a lot of logging information very quickly.
What does "Warning WorkingSet: Virtual Memory is . . . ." mean?
Unable To Export Large Objects To Excel After Upgrading to November 2017
AAALR greater than row applicator messages
AccessPoint slow to load due to too many files or folders mounted
We are looking for:
For example, we might see:
Information Server: Document Load: Beginning open of document Information System: Document Load - ODE1: Document \\path\\doc.QVW, AuthenLev(1). Authuser() Information DOC loading: Beginning load of document \\path\\doc.QVW. Warning WorkingSet: Virtual Memory is growing beyond parameters - 4.308(4.200) GB Warning WorkingSet: Virtual Memory is growing beyond parameters - 4.688(4.200) GB Warning WorkingSet: Virtual Memory is growing beyond parameters - 4.711(4.200) GB
There were no memory alerts prior to the load of doc.QVW, so we can start with this one.
Depending on our findings, we may then move on to:
Documents active during the day and while the problem is observed, can be individually reviewed for their basic memory footprint.
We do have some guidelines though that can be applied to make sure the server is configured for the best possible stability and performance.
Event Log Verbosity in Qlik View
Best setup setting for QlikView Server logging
How To Collect QlikView Server Log Files
Recommended Log files to analyze QlikView problem
QlikView Services and Log Guide