Qlik Community

Ask a Question

Support Knowledge Base

Search or browse our knowledge base to find answers to your questions ranging from account questions to troubleshooting error messages. The content is curated and updated by our global Support team

Announcements
Welcome to our newly redesigned Qlik Community! Read our blog to learn about all the new updates: READ BLOG and REPORTED ISSUES

Qlik Sense® Recommended Product Boundaries

Digital Support
Digital Support

Qlik Sense® Recommended Product Boundaries

Introduction

This Qlik Sense product boundaries guide is the result of tests performed, investigated and measured by the Qlik R&D Performance & Stability team.

Each boundary shown is the result of peak tests performed in a highly-specialized environment where a single metric has been tested separately, for each boundary, to establish a theoretical, stable maximum value. The values indicate what the Qlik Sense product is capable of for each individual boundary in isolation.  It is therefore important to treat each function boundary as a separate entity. The maximum values shown here may not necessarily be reproducible in other Qlik Sense environments and should only be used as general guidance when planning deployments. It is important take into consideration other factors and variables within your own environment such as; machine specification, size and number of apps, number of users, etc.

This guidance is intended solely for general informational purposes and its contents shall not be incorporated into, and do not form part of, the product documentation or any contract or agreement with Qlik.  The information regarding the products in this document are subject to change without notice.  ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS DOCUMENT, ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED.  Qlik does not represent, warrant, or guarantee any particular outcome or result and disclaims liability for any direct or indirect losses resulting from any reliance upon information contained or omitted in this document.  Qlik makes no commitment to deliver any future materials, code, or functionality and purchasing decisions should not be based upon any future expectation. 

This document constitutes confidential and proprietary information belonging to Qlik. The contents may not be disclosed to any person, organization, or entity, unless Qlik consents and such disclosure is subject to the provisions of a written nondisclosure and proprietary rights agreement, or intellectual property license agreement, approved by Qlik.  The distribution or availability of this document does not grant any license or rights, in whole or in part, to its content, the product(s), the technology, or intellectual property, described herein.

Boundary Criteria

The boundaries listed here were established by conducting tests where:

  • General bottlenecks such as CPU and RAM saturation were avoided
  • Bandwidth was 1Gbps
  • Modern SAS disks (connected through RAID controllers) were used
  • The services ran as expected (that is, without problems)
  • Selection response times were reasonable (<2s) and did not increase during the test

 

Tested on:


Qlik Sense June 2020 
Single Node
  • CPU: 2x Xeon E5-2690v2 @ 3.00 GHz
  • RAM: 384GB
  • OS: Windows Server 2012 R2

 

Maximum Theoretical Boundaries

The following table lists the maximum values for each individual boundary that can be theoretically used in Qlik Sense Enterprise.

Note: Functions listed here reaching their maximum values will have an impact on the maximum values that some of the other functions could reach.

 

 

Function

Maximum value (Theoretical)

New Users/Sessions per second 

7 (sustainable rate for 15 minutes with no errors)

Scheduled reloads per hour 

16000 reloads per hour / 2000 reloads per 15 minutes (with the execution results cleaning agent activated)

Data size 

2 billion (unique values per field)

Load script size (max) 

500000 rows

Sheets per app 

7500

Concurrent reloads 

Do not go above n(cores) – 2

Concurrent sessions 

18800 (Single node with default ephemeral ports setting)

Streams 

Owner of streams (data is uncached + streams are rendered in the hub):

  • 2,5s avg. resp. time: 40 streams

  • 3s avg. resp. time: 130 streams

User who does not own any stream (data is cached + streams are not rendered in the hub): 2.5s avg. resp. time: 1200 streams

 

 

Users / Sessions per second

A Qlik Sense single node deployment can handle a ramp-up of 7 users per second for a period of 15 minutes.

Adding more engine nodes behind a single proxy does not increase the ramp-up much as the ramp-up boundary is mainly related to authentication and session setup, which are handled by the proxy. For example, the user ramp-up pace for a four-node deployment with a single proxy service is more or less the same as for a single node deployment.

To get a higher ramp-up, proxy nodes rather than engine nodes should be added. This has been verified by adding an additional proxy service to a two-node deployment, which resulted in a ramp-up of 10 users per second.

 

Scheduled Reloads Per Hour

Tests conducted on the reload boundary for the Qlik Sense June 2020 release resulted in 16000 reloads per hour. This was tested in a five-node deployment with four reload nodes.

During the reload peak, the CPU usage of the master Qlik Sense Scheduler Service (QSS) on the central node was 10 - 15% on average.

Note: The test environment was dedicated to running small reloads in order to find the reload boundary. Actual results may vary depending on the deployed environment and other factors such as the size of the apps being reloaded, the number of users and the usage patterns.

Data Size

This is the practical maximum number of *unique* values per field that the Qlik Associative engine can load. (see https://community.qlik.com/blogs/qlikviewdesignblog/2012/11/20/symbol-tables-and-bit-stuffed-pointer...)

Load Script Size (max)

When testing the Qlik Sense load script limit, the result seems to depend on the web browser and perhaps the client machine rather than on Qlik Sense.

When testing with 500000 lines in a single section of a load script, the tested web browsers behaved as follows:

  • Google Chrome: Successfully accepted the script. The app could be reopened and scroll to the end of the script.
  • Microsoft Edge: Successfully accepted the script. The app could be reopened, but the browser failed when scrolling to the end of the script.
  • Mozilla Firefox: Successfully accepted the script. The web browser displayed performance messages.

The amount of RAM required per web browser differed significantly.

Sheets Per Application (“App”)

A single Qlik Sense app can contain up to 7,500 sheets and still have acceptable "Open app" response times for users who have access to 400 sheets or less.
In the tests performed, the average response times scaled close to linearly in the owner and cached scenarios, whereas in the uncached scenario, the response times scaled slightly worse than linearly.

Average App open Response.png

Average App open Response No sheets.png

Note: This is not a product boundary.  While it is possible to have more sheets in an app, it is not recommended from an end-user perspective as performance may be affected.

Disregarding response times, it is recommended to have no more than 3000 to 3500 sheets for personal use or visible to a user. Beyond that, the load becomes too heavy for the browser, which results in bad user experience.  Browser performance is dependent on the capacity of the client machine, and may be affected once the number of sheets reaches 6000 or more.

Concurrent Reloads

Qlik Sense scales up with reloads. As many reloads as needed can be started on a single reload node, but they will take longer and longer to finish as the node queues the reloads when busy. If the number of reloads to start exceeds the maximum number of parallel reloads allowed or the number of cores on the reload node, adding additional reload nodes will shorten the total duration of the reloads.
Note: The test conducted involved 100 apps of similar design. Actual results may vary based on the level of complexity in the apps.

  • Use reload nodes with the same CPU capacity: Additional reload tests have confirmed that reload nodes should have the same CPU capacity. This is because the scheduler load balancing is based on the server CPU. In unbalanced clusters, nodes with higher CPU capacity are loaded with more reload tasks than nodes with lower CPU capacity, which results in queues on the high capacity nodes and the low capacity nodes being idle earlier than the high capacity nodes. This means that the total reload time could be longer in unbalanced clusters than in balanced clusters.

In most cases, it is recommended not to allow more parallel concurrent tasks than there are cores: n(cores)-2 reloads per reload node
The reasons are as follows:

  • A reload task typically consumes at least one core.
  • The total time to complete a number of concurrent tasks that is larger than the number of cores does not improve with more parallel tasks running due to resource starvation of the server.
  • Each reload running is loaded into RAM. This means that the average RAM footprint grows with more parallel reloads.
  • With more parallel reloads than cores the probability for total starvation increases since the QIX engine is CPU hungry and other processes (for example, the operating system) might not get enough CPU cycles to perform their duties.

Concurrent Sessions:

This is the number of app sessions (that is, when you are not using the hub) that can be used at the same time.

Streams

It is the number of streams that a user has access to (and can see in the hub) that adds to the response times, not the total number of streams in the system.
The response times (including rendering) were fine up to having access to 40 streams in the hub. For example, the average response times were below 2.5 seconds for a user with access to 32 streams when there were 1000 streams in total in the system.
For users without access to any streams (except the ‘Everyone’ stream), the average response times scaled close to linearly when the data was cached. The response times (without rendering) were fine up to having access to 1200 streams in the system.

 
Related Content: 


1. https://blog.coeo.com/emmazambonini/2016/06/01/ephemeral-port-exhaustion

 

 

© 2018 QlikTech International AB. All rights reserved. Qlik®, Qlik Sense®, QlikTech®,and the QlikTech logos are trademarks of QlikTech International AB which have been registered in multiple countries. Other marks and logos mentioned herein are trademarks or registered trademarks of their respective owners

Labels (2)
Comments
Luminary
Luminary

Hi Sonja,

Where can we find these limits? I am interested to know the maximum number of streams. 

Kr,


Dion

Digital Support
Digital Support

Hello @dionverbeke 

The boundaries researched are documented inside this article. Streams are listed as well.

Note that, as mentioned in the doc, these are theoretical maximum numbers based on our team's tests. No hard caps exist.

Version history
Revision #:
1 of 1
Last update:
‎2020-11-11 04:54 AM
Updated by:
 
Contributors