Note: Our legacy performance testing offering can be found here: Qlik Sense Scalability Tools.

The Qlik Sense Enterprise Scalability Tools are used to create and run load and performance tests towards Qlik Sense Enterprise for Windows (QSEfW).

Included parts are:

• Documentation on how to use the package
• Command line tool for Windows, Mac and Linux. This command line tool (gopherciser) is open source and can be found here .
• GUI for Windows, Mac and Linux.

Downloadable files can be found here:
📎 Qlik Sense Enterprise Scalability Tools 

Qlik Sense documents to help analyze result and log files can be found here :https://community.qlik.com/docs/DOC-15451

Change log

v1.13.0

• Gopherciser v0.18.7 -> v0.20.1
  • QSEoW Nov24 support
  • Support sn-tabbed-container object
  • Openapp changed for Nov24 api compliance
  • New default websocket reconnect backoff pattern
  • Bugfixes

v1.12.0

• Updated scheduler
• Gopherciser v0.18.5 - v0.18.7
  • performance enhancements
  • new action stepdimension

v1.11.0 - v1.11.1

• Gopherciser v0.17.26 - v0.18.5
  • QSEoW Nov24 support
  • Support sn-tabbed-container object
    * Openapp changed for Nov24 api compliance
    * New default websocket reconnect backoff pattern
    * Bugfixes

v1.10.2 - v1.11.0

• Support for Feb 2024 Sense release
• Gopherciser v0.17.21 -> v0.17.26
  • Bugfixes
  • Report sub hypercube errors as errors instead of warnings (unless calculation condition)

v1.10.1 - v1.10.2

• Rebranded with new Qlik Logo

v1.10.0 - v1.10.1

• Gopherciser v0.17.7 -> v0.17.21
  • Added support for layout container object
  • Added support for new pivot table
  • Added support for sn-text object
  • Added support for sn-table object
  • Added support for sn-shape object
  • Added hyper cube sub-errors as warnings (These warnings will become errors in v1.11.0)

v1.9.0 - v1.10.0

• Gopherciser v0.17.7 -> v0.17.11
  • Open Hub Api compliance August 2023 release
  • Open App Api compliance August 2023 release

Your use of Qlik Sense Enterprise Scalability Tools will be subject to the same license agreement between you and Qlik for your Qlik Sense License. Qlik does not provide maintenance and support services for the Qlik Sense Enterprise Scalability Tools, however please check QlikCommunity for additional information on use of these products.

executing binary: ..........\Qlik Sense Enterprise Scalability Tools\resources/qsest-scheduler/scheduler.exe

When I run the script, I get the error : ERRORTypeError: Cannot read properties of undefined (reading 'ExecuteFunction')

failed to read JSON message: failed to read: use of closed network connection

Anyone know what is wrong 

Hello,  I'm trying to evaluate the performance of QlikSense apps using the Scalability Tool (QSEST), but I keep getting connectivity issues. Appreciate any suggestions on how to resolve the problem.

This package (referred to as Qlik Sense Scalability Tools) contains a complete set of tools for easy creation, execution and analysis of load/performance tests.

This tool is now deprecated and will not receive any further updates, please use the Qlik Sense Enterprise Scalability Tools instead.

Supported versions of Qlik Sense: all 2020, all 2021, 2022-aug

Included parts are:

• Standalone application for creating and executing a simulation script
• Documentation on how to use the package
• Regression analyzer
• Benchmarking package
• App evaluator package

QlikView and Qlik Sense documents to help analyze result and log files (previously included in this package) can be found here :https://community.qlik.com/docs/DOC-15451

Troubleshooting

For help to troubleshoot connection problems, please review Appendix A of the documentation or Connection Troubleshooting Tips

Change log

v5.17.0

• Add support for Qlik Sense May 2022 release
• Add support for Qlik Sense Aug 2022 release
  

v5.16.0

• Add support for Qlik Sense Feb 2022 release
  

v5.15.0

•  Add support for Qlik Sense Nov 2021 release

v5.14.0

• Add support for Qlik Sense Aug 2021 release

(See Readme.txt for changes in earlier versions of the tool.)

Your use of Qlik Sense Scalability Tool will be subject to the same license agreement between you and Qlik for your Qlik Sense License. Qlik does not provide maintenance and support services for the Qlik Sense Scalability Tool, however please check QlikCommunity for additional information on use of these products.

Hello testers, 

I am attempting to connect to a QS Enterprise instance with the QS Enterprise Scalability tools.  Could anyone please help me understand potential root causes and resolutions for the following error:

Version: v0.17.11 Failed to connect using 2 connection test functions. Error #1: failed to connect to engine over web socket: Failed connecting to sense server: Failed connecting to sense server: Error connecting to Sense: unexpected websocket scheme: "https" Error #2: HTTP request fail: Get "https://t***d/login": lookup t***d on xxx.xxx.xxx.xx1:53: server misbehaving

Could anyone please advise: do the QSEST (Qlik Sense Enterprise Scalability Tools)

1. Generate web sockets traffic against the Qlik Sense Proxy service? (top half of screen shot)
   OR
2. Generate API calles directly to the Qlik Sense Engine service?  (bottom half of screen shot)

And as a follow-up question, do the legacy QSST operate against the Proxy or the Engine?

Thanks,

JR

Hi everyone,

I'm looking for example applications from Performance Benchmarks Technical Brief. Do anyone know how can i find them? If they are unavailable, is there any official substitute for those?

Thank you

This page describes the settings for best performance for servers running the Qlik Associative Engine.
Latest update: May 2022

Windows 2022 has improved performance for servers with many physical cores. This table shows the definition used in the below document.

BIOS settings

The names of the settings and how to tune them may differ depending on the server manufacturer and model. Refer to the documentation for your server to find the equivalents of the settings listed above.

Operating system settings

Microsoft Windows

/ Cheers from the Scalability Center team

This package is deprecated and will no longer be maintained.

The document attached outlines all required information on how to access/execute/analyze results for the Qlik Hardware Benchmarking test package.

The goal with the HW Benchmarking package is to have a unified Hardware test that can be executed for various environments with the aim to establish a matrix for comparisons on how different servers and configurations perform with QlikView and/or Qlik Sense. This package can be used for analytics on your own to perform comparisons of performance for different hardware configurations.

The document contains information on how to download additional files needed. The package supports Qlik Sense Enterprise on Windows and QlikView Server 12 or newer.

This package is deprecated and will no longer be maintained.

This package (referred to as Qlik Sense Scalability Tool) contains a complete set of tools for easy creation, execution and analysis of load/performance tests.

Supported versions of Qlik Sense

For v2.x, v3.x use scalabilitytools-legacy.zip

Zip files named scalabilitytools-YEAR.zip supports all sense version for that year.

Note: Using windows built-in unzip functionality will not allow the .exe to launch unless unblocking zipped package first (properties -> unblock).

These packages exist only for legacy reasons to support load testing towards Sense v2.X-3.X. For any newer version of Qlik Sense please use the package located at: Qlik Sense Scalability Tools

Included parts are:

• Standalone application for creating and executing a simulation script
• Documentation on how to use the package
• Regression analyzer
• Benchmarking package
• App evaluator package

Your use of Qlik Sense Scalability Tool will be subject to the same license agreement between you and Qlik for your Qlik Sense License. Qlik does not provide maintenance and support services for the Qlik Sense Scalability Tool, however please check Qlik Community for additional information on use of these products.

The Qlik Sense Scalability Tool could not seem to work with either qlik-tabbed-container or tables using qlik-variable-input. Is this a known limitation? Or should I be doing something special so that the tool recognizes them?

Whenever I run the tool on sheets with either of these objects, the results do not include the data that's inside their tables. The execution shows a warning reading "No datapages found in layout for object..." for both objects. In addition, when I try to use the "Select" action to click on a tab (within a qlik-tabbed-container) or a button (within a qlik-variable-input), I get a warning reading "selection in object visualization qlik-tabbed-container not yet supported" or "selection in object visualization qlik-variable-input not yet supported".

If there is no workaround at this stage, is there a plan to include support for these in any upcoming release?

This package (referred to as QVScalabilityTools) contains a complete set of tools for easy creation, execution and analyzing of load/performance tests and regression tests.

QlikView documents to help analyze result and log files(This was previously included in this package):Scalability Results Analyzer

Older versions of the tool can be found here: Tool for easy creation of load/performance tests of QlikView (v.10 and 11)

Tools for Qlik Sense can be found here: Qlik Sense Scalability Tools

Note: Using windows built-in unzip functionality will not allow the .exe to launch unless unblocking zipped package first (properties -> unblock).

Note: Regarding Apache Log4j, reference CVE-2021-44228 (also referred to as Log4Shell), these set of tools are not directly vulnerable to the exploit, however JMeter, the tool required as a separate download, is. How to mitigate the vulnerability for JMeter has been documented here : https://community.qlik.com/t5/Knowledge/Qlik-Scalability-tools-and-log4j/ta-p/1872009 

Included parts are:

• Standalone application for creating a JMeter script
• Support files for launching the script by utilizing the JMeter engine (JMeter installed is a pre-requisite)
• Documentation on how to use the package

Version 1.2.7->1.2.8

• Changes Section Access requests

Changelog 1.2.6->1.2.7

• NTLM support removed due current solution broken and lack of a new secure solution.
• Small updates for static content, for future product releases.

2019-08-26

Documentation updates due to:
- New recommended JMeter version (4.0)
- Changes to the licensing for the Oracle Java implementation (seehttps://www.oracle.com/technetwork/java/javase/overview/oracle-jdk-faqs.html)
- Open-source Java Development Kits (JDKs) have been verified for use with JMeter and the QlikView Scalability Tools

Changelog 1.2.5->1.2.6

• Added support for QlikView 12.50 and 12.20SR10

Changelog 1.2.4->1.2.5

• Added support for QlikView 12.30

Changelog 1.2.3->1.2.4

• Added support for JMeter 4.0 and Java 9.
• Bugfixes for regression testing
• Added actions: Select Excluded, Select All, Select Possible
• Added support for 12.20 SR3

Changelog 1.2.2->1.2.3

• Bugfix for export to excel action
• Added transfer state (bookmark) support for document chaining
• GUI: Template selection more accurately show supported Qv versions

Changelog 1.2.1->1.2.2

• Bugfixes, make sure minimum amount of selections are respected in certain corner cases

Changelog 1.2->1.2.1

• Improved performance for high throughput scenarios

Changelog 1.1 -> 1.2

• Improved assertions logic to show faulty actions as errors
• Change JMeter template scripts to use groovy instead of beanshell
• New prerequisites:
  • JMeter version 3.0
  • Java 64-bit JRE 8

Your use of QVScalabilityTools will be subject to the same license agreement between you and QlikTech for QlikView Server. QlikTech does not provide maintenance and support services for QVScalabilityTools, however please check QlikCommunity for additional information on use of these products.

PLEASE NOTE:

The supported versions of Java are 8 and 9. Supported version of JMeter are 3.0 and 4.0, but not 3.2

I put together check lists as a help troubleshooting connection settings problems when using Qlik Sense Scalability Tools.

Using header auth

1. Is the app published?
2. Can you connect manually to the virtual proxy with header auth using modify header plugin in firefox, fiddler or similar.
   1. If it works, make sure the user which get logged in is the one from the header not your ntlm user.
   2. Can the header user access the app you want to simulate?
3. If manual connection (1) with header works, make sure all these settings correspond to your tool settings
   1. VirtualProxyPrefix settings same as Prefix defined in QMC on the virtual proxy
   2. HeaderName same as Header authentication header name on the virtual proxy
   3. ConnectionType set to Header.
   4. Server is set to the same server name as the certificate, default fully qualified domain name, without defining protocol etc, i.e. myserver not https://myserver/hub.
   5. A default sense installation uses https and a self-signed certificate. Make sure the SSL checkbox is checked when using SSL or TLS (default) and the AllowUntrustedCertificate checkbox is checked if using the self-signed or any other “untrusted” certificate.
4. If Connect to App doesn’t work, try doing a simple scenario of 1 user 1 iteration with only a small timerdelay in the scenario, then execute this using the executor (i.e. not connect to app). Does this get any errors?
5. If you are getting license access denied, make sure the header users have an appropriate amount of tokens allocated.

Using NTLM auth

1. Make sure the user has access to the app by manually connecting to it.
2. If manual connection (1) with header works, make sure all these settings correspond to your tool settings
   1. VirtualProxyPrefix settings same as Prefix defined in QMC on the virtual proxy or blank if using root (default)
   2. ConnectionType set to NTLM.
   3. Server is set to the same server name as the certificate, default fully qualified domain name, without defining protocol etc, i.e. myserver not https://myserver/hub.
   4. A default sense installation uses https and a self-signed certificate. Make sure the SSL checkbox is checked when using SSL or TLS (default) and the AllowUntrustedCertificate checkbox is checked if using the self-signed or any other “untrusted” certificate.
3. If Connect to App doesn’t work, try doing a simple scenario of 1 user 1 iteration with only a small timerdelay in the scenario. Does this get any errors?
4. If you are getting license access denied, make sure the header users have an appropriate amount of tokens allocated.

We have a multinode QlikView server with  4QVWS, 6 QVS, 2 QDS and 2 File share servers. I am running scalability for 700 concurrent users based on complex scenarios on the 3 most heavily used dashboard in Production. I am still not able to find out the breaking point of my platform. 

Has anyone tried to achieve this before? Any leads will be appreciated.

Hi Daniel,

Scalability testing With Jun 2020 version not working getting  warnings Open App failed to get proxy session name.

However it seems working for April 2020 and lower versions.

Can you please help me to fix this. Proxy settings looks fine if i add RIM servers instead DNS name its working.

Thanks.

I am trying to select from a listbox in a Qlik Sense app with Type "Values", but I can't seem to hit the entry I need to select. Could someone please help point me in the right direction?

The listbox has five possible values. So in the Qlik Sense Scalability Tool's Select action I have specified Type=Values, and for Values I've listed 0, 1, 2, 3,4.  The qElemNumber of what I want to select is 1, so I specify Dimension=1 in the tool. When I execute the script however, I get a "Selection in object failed".

I saw another thread in this forum touching on how to determine these values in QV (https://community.qlik.com/t5/Qlik-Scalability/Need-to-select-2-static-selections-from-Listbox-in-QVScalability/gpm-p/132068), but is there a way to do something similar in Qlik Sense?

Ultimately I just need to select one entry in this listbox on a consistent basis. I've tried using SelectString using the qText value of the entry but that also fails.

I am running into a problem where NTLM authentication from the QV Scalability Tool fails with a 401, but I am able to access the document normally from Chrome.

Is there any additional config that I need to do apart from ticking the NTLM box?

From the ScriptGenerator the NTLM box is ticked, and I have a simple scenario that simply opens a QV document. The source XML looks like this (actual document URL value blocked out with ####):

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<script method="JMeter" type="RegressionScript" qlikview="12" GUIversion="1.2">
<settings>
<document url="####" />
<thread duration="-1" loopcount="1" rampup="1" threadcount="1" />
</settings>
<variables>
<set value="..\Analyzer\JMeterExecutions\{SCRIPTNAME}_{HOSTNAME}_[{THREADCOUNT}-{RAMPUP}-{LOOPCOUNT}-{DURATION}]_${__P(DT)}.txt" name="LOGFILE" />
<set value="..\SupportFiles\seedfile1.txt" name="STANDARDRAND" />
<set value="..\SupportFiles\seedfile2.txt" name="CHARTRAND" />
<set value="" headers="" name="HEADERFILE" />
<set value="..\SupportFiles\doclist.txt" usereduced="FALSE" name="USEREDUCEDDOCS" />
<set value="" sectionaccess="FALSE" name="USESECTIONACCESS" />
<set value="" usebasicauth="FALSE" name="BASICAUTHENTICATION" />
<set value="" dosavebinaries="True" name="DOSAVEBINARYFILES" />
</variables>
<scenario>
<generic action="Access Point" label="" />
<generic action="Open Document" label="" />
<generic action="Timer Delay - UniformTimer" min="30" max="35" />
</scenario>
</script>

Once I execute this however, I end up with a 401 during authentication:

1590110384642,20,/QvAJAXZfc/Authenticate.aspx,401,Unauthorized,QlikView 1-1,,false,140,1,1,20

Consequently the resultant regression results file is empty (private data blocked out with ####):

<?xml version="1.0" encoding="UTF-8" ?>
<testresult testname="####" QVSversion= "NoVersion" Document="####" timeStamp="2020-05-22-111141">
	<requestedobject action="Open Document with SA #2" id="####" inputValue="" threadIteration="1" testLoop="0">
	</requestedobject>
</testresult>

 I've been trying to get this to work for some time now so any help would be greatly appreciated.