We're experiencing a performance issue with a Qlik Sense application. We've been working on this for weeks, tweaking the data model, expressions, etc., but it doesn't seem to be enough. At this point, we suspect the issue might be server-related.
Our architecture is based on a cluster:
A central server distributes the workload to other nodes. Another node is dedicated to reload tasks. Two other nodes are dedicated to running Qlik Sense applications. These two servers are:
Server 1: Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz (3.4GHz with Turbo Boost), 2 processors, 32 cores, 64 threads, 512GB RAM. Server 2: Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz (3.5GHz with Turbo Boost), 2 processors, 14 cores, 28 threads, 256GB RAM.
The application causing us trouble is the largest one we have (80+ million rows in the fact table) and also the most heavily used. We've set up a Load Balancing rule to run this application on Server 1, the more powerful machine, while the rest of the applications run on Server 2.
Strangely, we've noticed that this large application performs better on the smaller server than on the larger one. We can't ignore the impact of concurrency: the number of users accessing the large application is significantly higher than those using the smaller ones. So, when I run my tests, I might have the smaller server practically to myself, while on the larger server, I'm sharing resources with dozens of users navigating through the application. However, we’ve been monitoring CPU usage during these tests, and the large server’s CPU utilization remains well below 100%.
So, I have two questions:
1) Can we rule out high user concurrency as the cause of the worse performance on the large server, given that CPU usage doesn’t seem to be excessive? 2) Are there any server configurations we should check to ensure we’re getting the best possible performance?
80 M fact-records isn't small but also not really large or even huge. In regard to the mentioned hardware I would expect a quite sufficient performance.
Therefore I suggest to check the application again for optimizations by developing the data-model in regard to a star-scheme, splitting high cardinality fields like timestamps into dates and times, transferring all essential logic into the data-model and using simple expressions like: sum(Field) or count(Field) - no (nested if-loops, aggr(), interrecord-functions).
If you suspect the servers as the main issue, I'd recommend searching for Qlik technical information on recommended server configurations. It's could be easier to find, but here is an example of such documentation, which is 5 years old:
Maybe you can dig deeper and find something more recent.
If you still need to troubleshoot your application performance, I'd be happy to perform a proper Performance Tuning exercise on your large app - this is one of my specialties and one of the services that I provide to clients that need to improve their application performance.
Have you tried tuning the application using QSDA Pro? QSDA Pro will help you identify and resolve performance issues in the app. My best guess is that you have a number of multi-table aggregations in the app.
Let me know if you can answer any questions about QSDA Pro or arrange a POC for you.
