Server response time increases during loads

[rogeriobarbosa]

Hello, I have the following problem, during my loads the Qlilview server gets slow on the network, running from the ping command 10.1. *. * It increases the response time considerably, can anyone help me?

[Peter_Cammaert]

Probably due to RAM exhaustion.

Connect to your server (by way of RDP), start task manager, select the Performance tab and watch it when the server starts to reload. What is the peak memory usage in %.

[rogeriobarbosa]

Hello, Memory usage is at 30%, the processing of the 90% peaks at that time the response time on the network gets very high

[Peter_Cammaert]

That means that your server has not enough RAM to run both QVS (with a number of documents loaded) and the reload tasks you have scheduled. Since the reloads are actually sort-of blackbox operations without user interface and no human waiting for them to finish in a certain amount of time, you will most likely experience the negative effects of this collision in QVS response times: the QlikView Access Point will become sluggish.

Note that every reload task is a separate process that will try to allocate its own RAM blocks. Fortunately at the end of the reload, RAM for a particular task will be released, so this 90% situation is not permanent.

There is one quick solution that won't cost you more than a bit of energy and time: reschedule all reloads to a window in which you don't expect your users to be on-line.

But there is only one real, long-term  solution: buy and install more RAM.

Intermediate solutions include optimizing your documents (QVS uses less RAM) or streamlining your load scripts, but those come without any guarantees whatsoever to mitigate the 90% situation.

Best,

Peter

[rogeriobarbosa]

Hello Peter, my server has 192 RAM, I am not using nor 30% of its capacity, the problem occurs during the loads my processor reaches 90% sometimes with peak of 100%, at that time my server response time in Network gets high when I ping. 10. * 1. * 1. * it takes time to respond causing slowness.

[Peter_Cammaert]

Indeed, the situation you are descrbing is what I explained in my previous post. No I'm not a psychic; everybody who runs QlikView servers gets in this fight between QlikView Server and Distribution Service sooner or later.

In other words (and based on what information you provided in this thread, correct me if I'm making the wrong assumptions):

• Imagine your QVS has stabilised and is using 30% of available RAM (on average). This may grow, and because QVS never releases memory, it will only grow. A service restart will reset this RAM usage to a low level.
• QDS (also a service which means it's always present in RAM) checks the task schedule and detects a task that should run.
• QDS allocates a new reload engine (if one is available, see QMC): and tells it to reload a particular document.
• Depending on the size of the tables and the complexity of the load script (JOINs? STOREs?), this reload engine will start requesting memory from Winbdows outside of the RAM that is in use by QVS. The reload engine is a regular Windows process that runs in the background. At any moment, a reload engine reloads a single document and executes a single load script. When the reload engine is done executing the script, it saves the document, releases all memory and disappears.
• If the QDS decides to run 4 reloads in parallel, 4 reload engines will be started and they will all try to allocate as much RAM as needed. They cannot grab RAM from the QVS (very asocial, that one) so they'll have to do with whatever remains.
• Eventually, all your 192GBytes are gone and RAM is 100% allocated. No big deal for the QVS, it has all RAM it needs (not really, but you get the idea). But the reload engines that still need more RAM will keep asking for it and Windows will start swapping memory blocks to disk in order to free up some memory. This is extremely slow and extremely expensive. The reload engines which are highly I/O dependent will slow down to a crawl, and at the same time Windows will slow down as well (swapping is a high-priority activity). Your disks will start to glow

There you have it. The reloads won't crash. But they will take even more time to complete because of the swapping. And your QVS will become sluggish, if not completely unresponsive.

This unpredictable memory consumption is the main reason why in high-load environments a Publisher is usually run on a separate machine.