QlikSense CPU usage hits 100% while loading data from QVD

[Nims]

While running below load statement, CPU Utilization reaches 100% while RAM Usage shows 20%. I am using very high spec server. Also I am running a simple load statement. I am wondering why CPU reaches it's maximum while running simple load statement. Can anyone help me with this. 

ValueEntry:
LOAD
*
FROM [lib://QVDs (qlik_qlick)/ValueEntry_Full.qvd](qvd);

[rohitk1609]

Can you specify how much size of concern qvd and what are the specification of server?

[Nims]

Server has 1 TB RAM and 60 Core CPU. The QVD size is 17.6 GB.

[anderseriksson]

That's a nice server!
In generell when Qlik executes a load statement it uses as much resources as it can utlilize in order to cut the loading time short.
Your load statement is very simple and thus Qlik can use parallell processing to speed things up.
There is no dependency between rows or such, simple brut force will make the load quick.
There is no need for any more memory than what the qvd-file will occupy in memory.
If the load were restricted to only use one CPU (no parallelling) the load would take longer time but not use as much resources simultanusly. 
The workload of the load-statement will be pretty much the same,
only question is if it is to be finished as fast as possible (peek CPU) or
use as little resources as possible (only one CPU but for much longer time).
Normal for Qlik is to run as fast as possible with the resources available.

[Nims]

Thank you Anderseriksson for your response.

But my concern is on performance. Data reload is scheduled for every 2 hours. Hence users are facing calculation time out error while accessing reports as CPU utilization is hitting 100% while reload.

Is there any solution to overcome this issue?  As per my understanding, Qlik utilizes RAM during data reload . But in this scenario, CPU is utilized completely even though server has enough RAM.

[anderseriksson · Accepted solution]

Naturally Qlik uses RAM during reload, as much RAM as it needs to hold the data being reloaded.
Qlik also uses CPU when reloading, as much CPU as it can get access to and utilize in order to complete the reload in as short period of time as possible.
RAM is working storage, CPU is the working power, Qlik needs both.
I'm not sure how you best would limit the CPU load during reload in order to release more CPU for the end-users.
Usual procedure is to have two servers, one back-end doing the reloads and one front-end serving the end-users. That way the reloads are separated from the end-users.
You can limit the CPU usage of the Engine by using throttling but that would affect the users also.
Throttling is to allow other processes than Qlik on the server that would otherwise be starved when Qlik uses all CPU.

[anderseriksson]

One thing you can do is limit the number of concurrent reloads in the scheduler.
Running several reloads at the same time will take more CPU.
On the other hand with less concurrent reloads you might experience the reloads take longer to complete as they will be queued after each other.
It all depends on how the scheduling of tasks look on your server.

[Nims]

Thank you Anderseriksson for your inputs

[PaulinaWanago]

If Qlik use all available CPU it means that lower CPU number in QMC is applied then reloads will run faster?

I have 10 CPU core and applied in QMC only 5. If I increase the CPU core in QMC to 8 the more task will run but it means that this reloads will run slower?

[anderseriksson]

Are you confusing Cores with number of parallell Tasks?
The more CPU cores you have available the faster the reloads will run.
The more parallell tasks you allow the slower each reload will run.
Fewer parallell reloads the ones that get started right away will run faster but
instead some will get queued and instead run after the first ones are finished.
Aka either you run 10 reloads directly in parallell or you run 5 first and queue 5 to run after those.
Allowing too many reloads in parallell you can choke the server.
Limiting the number of parallell reloads you can spread them over a longer period to avoid that.
(but instead you run the risk of the queued reloads to time out before starting)
What is right for your server all depends on the the resources available and the size and complexity of your data and apps.