Skip to main content
Announcements
See why Qlik is a Leader in the 2024 Gartner® Magic Quadrant™ for Analytics & BI Platforms. Download Now
cancel
Showing results for 
Search instead for 
Did you mean: 
Nims
Partner - Contributor II
Partner - Contributor II

QlikSense CPU usage hits 100% while loading data from QVD

While running below load statement, CPU Utilization reaches 100% while RAM Usage shows 20%. I am using very high spec server. Also I am running a simple load statement. I am wondering why CPU reaches it's maximum while running simple load statement. Can anyone help me with this. 

 

ValueEntry:
LOAD
*
FROM [lib://QVDs (qlik_qlick)/ValueEntry_Full.qvd](qvd);

 

Nims_1-1650134804002.png

 

Labels (1)
1 Solution

Accepted Solutions
anderseriksson
Partner - Specialist
Partner - Specialist

Naturally Qlik uses RAM during reload, as much RAM as it needs to hold the data being reloaded.
Qlik also uses CPU when reloading, as much CPU as it can get access to and utilize in order to complete the reload in as short period of time as possible.
RAM is working storage, CPU is the working power, Qlik needs both.
I'm not sure how you best would limit the CPU load during reload in order to release more CPU for the end-users.
Usual procedure is to have two servers, one back-end doing the reloads and one front-end serving the end-users. That way the reloads are separated from the end-users.
You can limit the CPU usage of the Engine by using throttling but that would affect the users also.
Throttling is to allow other processes than Qlik on the server that would otherwise be starved when Qlik uses all CPU.

View solution in original post

7 Replies
rohitk1609
Master
Master

Can you specify how much size of concern qvd and what are the specification of server?

Nims
Partner - Contributor II
Partner - Contributor II
Author

Server has 1 TB RAM and 60 Core CPU. The QVD size is 17.6 GB.

anderseriksson
Partner - Specialist
Partner - Specialist

That's a nice server!
In generell when Qlik executes a load statement it uses as much resources as it can utlilize in order to cut the loading time short.
Your load statement is very simple and thus Qlik can use parallell processing to speed things up.
There is no dependency between rows or such, simple brut force will make the load quick.
There is no need for any more memory than what the qvd-file will occupy in memory.
If the load were restricted to only use one CPU (no parallelling) the load would take longer time but not use as much resources simultanusly. 
The workload of the load-statement will be pretty much the same,
only question is if it is to be finished as fast as possible (peek CPU) or
use as little resources as possible (only one CPU but for much longer time).
Normal for Qlik is to run as fast as possible with the resources available.

Nims
Partner - Contributor II
Partner - Contributor II
Author

Thank you Anderseriksson for your response.

But my concern is on performance. Data reload is scheduled for every 2 hours. Hence users are facing calculation time out error while accessing reports as CPU utilization is hitting 100% while reload.

Is there any solution to overcome this issue?  As per my understanding, Qlik utilizes RAM during data reload . But in this scenario, CPU is utilized completely even though server has enough RAM.

anderseriksson
Partner - Specialist
Partner - Specialist

Naturally Qlik uses RAM during reload, as much RAM as it needs to hold the data being reloaded.
Qlik also uses CPU when reloading, as much CPU as it can get access to and utilize in order to complete the reload in as short period of time as possible.
RAM is working storage, CPU is the working power, Qlik needs both.
I'm not sure how you best would limit the CPU load during reload in order to release more CPU for the end-users.
Usual procedure is to have two servers, one back-end doing the reloads and one front-end serving the end-users. That way the reloads are separated from the end-users.
You can limit the CPU usage of the Engine by using throttling but that would affect the users also.
Throttling is to allow other processes than Qlik on the server that would otherwise be starved when Qlik uses all CPU.

anderseriksson
Partner - Specialist
Partner - Specialist

One thing you can do is limit the number of concurrent reloads in the scheduler.
Running several reloads at the same time will take more CPU.
On the other hand with less concurrent reloads you might experience the reloads take longer to complete as they will be queued after each other.
It all depends on how the scheduling of tasks look on your server.

Nims
Partner - Contributor II
Partner - Contributor II
Author

Thank you Anderseriksson for your inputs