question about Max number of simultaneous QlikView engines for distribution - database bottlenecks crash QDS requiring hard reboot of QlikView Server

[daveatkins]

Trying to troubleshoot a recent problem which appears to be related to long-running queries on database. There are hundreds of applications and dozens of databases, so have not yet narrowed down, but what happens is:

1) DBAs report that non-QlikView related queries are running for 24-48 hours on one server resulting in big performance degradations.

2) QlikView distribution servers stops working...cannot be terminated from services panel...have to perform a reboot to get it back up. Reload tasks running much longer than normal. This contributes to the database backlog also.

My question is about this setting that we have not thought about in many years:

Max number of simultaneous QlikView engines for distribution

I want to verify if we have set it reasonably or if it could be too high...and how to set it properly.

QlikView server is:

Client Build Number	12.70.20200.0
Target Platform	x64
QlikView Build Info	release/QV12-70/48

Hardware is 32-cpu

Operating System Version

Win32_OperatingSystem=@ X64 (Microsoft Windows NT 6.2.9200.0)

Physical Memory

262109Mb

settings are:

settings are Max number of simultaneous QlikView engines for distribution:13 

Max number of simultaneous QlikView engines for administration: 8

[Chip_Matejowsky]

Hi @daveatkins,

If you’re not already aware, the Qlik white paper Scaling the QlikView Publisher is a great resource for determining the configuration of QDS settings. You may need to optimize some of the settings within the Settings.ini file of the QDS and the QMS, and the white paper provides that.

Also, the QlikView Governance Dashboard (QVGD) is a utility that can help you to better administer your QV Server/Publisher environment and has a sheet with a heat map graph of your reload/distribution tasks.

Suggest that you start with these two things and ping back if you have further questions.

Best Regards

[marcus_sommer]

Beside any server-settings I suggest to consider to slice the tasks into smaller chunks by applying a multi-tier data-architecture with (more) incremental approaches.

[daveatkins]

can someone just say whether these settings are reasonable or not? Server has run fine for years until last few days; I just need to know if I should consider adjusting these settings. Would these setting cause the server to hang if reloads were being delayed due to sql server queries taking 4-6 hours to complete? Priority has to be on finding out why the sql server is having issues; just need to know if I should spend 10 minutes on Qlik or if that could make it worse.

[marcus_sommer · Accepted solution]

Long running tasks could increase the RAM consumption of the distribution engine quite significantly and you may end by a lack of RAM for QlikView and/or the entire server which is often causing an unstable system. Therefore you may take at first a look on the RAM consumption during the tasks-execution.

Beside the settings for the number of max. parallel tasks-execution there are more settings to define various timeouts and for the queuing (max. consumption of CPU + RAM) and probably some more. But in the defaults these settings are quite well balanced and and shouldn't be to easily adjusted - at least not in a more or less trial and error approach without knowing what exactly caused the issues. Means not to cure the impact else the causes.

Beside restarting the QlikView server you may also restart the data-base environment and the storage/network system to exclude to look for any temporary stuff. Further I would check the data-base and network performance before diving deeper on the QlikView side.

[Chip_Matejowsky]

Hi @daveatkins,

Every QlikView Server/Publisher environment is different, so I typically wouldn't want to make architectural statements unequivocally. It seems you have diagnosed the issue to be on the data source side so if that is the case then would suggest you focus on that. 

Best Regards 

[daveatkins]

Thanks; this is helpful. Focusing efforts on the databases now. One issue we uncovered was an end user has scheduled a report to run every day and extract 6 years of data into a file! DBAs now empowered to "terminate with extreme prejudice" when queries are found running > 4 hours...