How many Parallel Reloads can I run on QV server?

[nickjose7]

Hi,

I am working on QV Server 11.20 SR9 with the following config:

I see currently I am only able to run 4 Parallel reloads. How do I increase that and upto what limit can it be done?

Thanks

Nick

[Miguel_Angel_Baeyens]

The recommendation is set the maximum to 9 or number of cores -1. In your case, 4 is the highest number of cores, so 4 is the maximum number of tasks you can have running at any given time.

Technically, as long as memory permits, you can have as many tasks as cores, but leaving 1 core for the operating system and other activities makes sense.

With version QlikView version 12, tasks are triggered until a certain CPU and RAM threshold is reached, after which the task is queued until more resources are available. You will need to count on that as well.

[nickjose7]

Thanks Miguel.

Would you mind sharing your views on this query also: Has anyone upgraded QV Server from 11.20 SR9 to QV November 2017??

[nickjose7]

Don't I have 16 cores as shown in Task Manager:

[Miguel_Angel_Baeyens]

Yes, you do. I don't know where I took the 4 from.

So 15 concurrent tasks maximum, 9 (recommended) or depending on the concurrency and utilized resources.

[nickjose7]

Thanks a lot Miguel.

Here are some more details of the two servers we have (besides test server):

Server A (with 32 Cores)	Server B (with 16 Cores)
QlikviewDirectoryServiceConnector	QlikViewDistributionService
QlikviewManagementService
QlikviewServer
QlikviewWebserver

This is the setting I currently see on QMC on Server A:

Could you please finally suggest the right changes I should make and WHERE to increase the no. of reloads for both manual as well as automated tasks, on the basis of above actual details.

Many thanks for your continued help.

-Nick

[Miguel_Angel_Baeyens · Accepted solution]

The main service that matters for reloads is the QlikView Distribution Service, in your case, Server B.

Any task, automated (scheduled) or manually triggered (clicking on the run button) counts towards the "Maximum number of QlikView engines for distribution", currently set to 4 according to the QMC screenshot above. I would not increase it above 8, since given how intensively QlikView 12 uses resources during reloads, some of those 8 might be even queued.

It is important to understand the whole picture. Knowing the hardware is important, but it is equally important to know how long your tasks run, usually, and how complex those tasks are, and how many of them can be run in parallel.

To fine tune the resources you are willing to give to QlikView Distribution service, you can adjust the values of these settings in the file QVDistributionService.exe.config in the Program Files folder:

• Memory: <add key="MemoryOverloadLimit" value="90" />
• CPU: <add key="CpuOverloadLimit" value="75" />

These are the values by default and they work in combination. Assuming the default settings, when the server RAM is above 90% or CPU is above 75% no more tasks will run, and if any is triggered, it will appear as "Queued" until those values are below the number specified.

That's why increasing the parallel tasks up to 15 (all cores -1) would not make complete sense if, say, 5 of them already consume above those settings.

Also, because if the tasks running are resource intensive, the rest of the tasks running will compete for such resources, resulting in an increased reloading time, the opposite you want to achieve. Of course, if you already measured your Server B activity and it never goes above 90% RAM or 75% CPU, you can safely increase the number of concurrent tasks in the QMC.

Last but not least, remember to modify the Windows Registry should you want to set the concurrent tasks above 8 or 9. Check the Qlik Support article https://qliksupport.force.com/articles/000026361 and if you don't have access to the Support Portal, ask your closest Partner or Qlik to send it to you.

[nickjose7]

Thanks a lot mbaeyens  for a very comprehensive answer. Marking it correct.

Just one last query - As I am still using QV 11.20 SR9 not QV 12, Should I still not increase the reloads more than 8?

[Miguel_Angel_Baeyens]

QlikView 11.20 is not so exhaustive in resource consumption in general, and there are some other options to consider for the reloads, like CPUAffinity (discontinued in version 12) instead of the CPU and RAM thresholds. Ask Qlik Support for the article related to this setting for the QlikView Distribution Service.

While the reloads are individually slower in 11.20 compared to 12, you will need to see how many of them the server can handle, and their resource consumption in concurrency is equally predictable: the simpler the tasks the more the server will be able to run at the same time.

In any case, and assuming the server is only running the operating system and QlikView Distribution service, leaving the number of simultaneous tasks 2 values below the number of cores is always a good safety net for whatever peak load the server might experience. Antivirus, backup software, monitoring software which are running at the same time also need resources and could eventually compete for those if all threads are being used by QlikView.

[nickjose7]

Thanks a lot mbaeyens‌. Grateful for your extended help.