Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
While running below load statement, CPU Utilization reaches 100% while RAM Usage shows 20%. I am using very high spec server. Also I am running a simple load statement. I am wondering why CPU reaches it's maximum while running simple load statement. Can anyone help me with this.
ValueEntry:
LOAD
*
FROM [lib://QVDs (qlik_qlick)/ValueEntry_Full.qvd](qvd);
Naturally Qlik uses RAM during reload, as much RAM as it needs to hold the data being reloaded.
Qlik also uses CPU when reloading, as much CPU as it can get access to and utilize in order to complete the reload in as short period of time as possible.
RAM is working storage, CPU is the working power, Qlik needs both.
I'm not sure how you best would limit the CPU load during reload in order to release more CPU for the end-users.
Usual procedure is to have two servers, one back-end doing the reloads and one front-end serving the end-users. That way the reloads are separated from the end-users.
You can limit the CPU usage of the Engine by using throttling but that would affect the users also.
Throttling is to allow other processes than Qlik on the server that would otherwise be starved when Qlik uses all CPU.
Can you specify how much size of concern qvd and what are the specification of server?
Server has 1 TB RAM and 60 Core CPU. The QVD size is 17.6 GB.
That's a nice server!
In generell when Qlik executes a load statement it uses as much resources as it can utlilize in order to cut the loading time short.
Your load statement is very simple and thus Qlik can use parallell processing to speed things up.
There is no dependency between rows or such, simple brut force will make the load quick.
There is no need for any more memory than what the qvd-file will occupy in memory.
If the load were restricted to only use one CPU (no parallelling) the load would take longer time but not use as much resources simultanusly.
The workload of the load-statement will be pretty much the same,
only question is if it is to be finished as fast as possible (peek CPU) or
use as little resources as possible (only one CPU but for much longer time).
Normal for Qlik is to run as fast as possible with the resources available.
Thank you Anderseriksson for your response.
But my concern is on performance. Data reload is scheduled for every 2 hours. Hence users are facing calculation time out error while accessing reports as CPU utilization is hitting 100% while reload.
Is there any solution to overcome this issue? As per my understanding, Qlik utilizes RAM during data reload . But in this scenario, CPU is utilized completely even though server has enough RAM.
Naturally Qlik uses RAM during reload, as much RAM as it needs to hold the data being reloaded.
Qlik also uses CPU when reloading, as much CPU as it can get access to and utilize in order to complete the reload in as short period of time as possible.
RAM is working storage, CPU is the working power, Qlik needs both.
I'm not sure how you best would limit the CPU load during reload in order to release more CPU for the end-users.
Usual procedure is to have two servers, one back-end doing the reloads and one front-end serving the end-users. That way the reloads are separated from the end-users.
You can limit the CPU usage of the Engine by using throttling but that would affect the users also.
Throttling is to allow other processes than Qlik on the server that would otherwise be starved when Qlik uses all CPU.
One thing you can do is limit the number of concurrent reloads in the scheduler.
Running several reloads at the same time will take more CPU.
On the other hand with less concurrent reloads you might experience the reloads take longer to complete as they will be queued after each other.
It all depends on how the scheduling of tasks look on your server.
Thank you Anderseriksson for your inputs