Qlik Sense Enterprise Scalability Tools

[Daniel_Larsson]

Note: Our legacy performance testing offering can be found here: Qlik Sense Scalability Tools.

The Qlik Sense Enterprise Scalability Tools are used to create and run load and performance tests towards Qlik Sense Enterprise for Windows (QSEfW).

Included parts are:

• Documentation on how to use the package
• Command line tool for Windows, Mac and Linux. This command line tool (gopherciser) is open source and can be found here .
• GUI for Windows, Mac and Linux.

Downloadable files can be found here:
📎 Qlik Sense Enterprise Scalability Tools 

Qlik Sense documents to help analyze result and log files can be found here :https://community.qlik.com/docs/DOC-15451

Change log

v1.12.0

• Updated scheduler
• Gopherciser v0.18.5 - v0.18.7
  • performance enhancements
  • new action stepdimension

v1.11.0 - v1.11.1

• Gopherciser v0.17.26 - v0.18.5
  • Bugfixes

v1.10.2 - v1.11.0

• Support for Feb 2024 Sense release
• Gopherciser v0.17.21 -> v0.17.26
  • Bugfixes
  • Report sub hypercube errors as errors instead of warnings (unless calculation condition)

v1.10.1 - v1.10.2

• Rebranded with new Qlik Logo

v1.10.0 - v1.10.1

• Gopherciser v0.17.7 -> v0.17.21
  • Added support for layout container object
  • Added support for new pivot table
  • Added support for sn-text object
  • Added support for sn-table object
  • Added support for sn-shape object
  • Added hyper cube sub-errors as warnings (These warnings will become errors in v1.11.0)

v1.9.0 - v1.10.0

• Gopherciser v0.17.7 -> v0.17.11
  • Open Hub Api compliance August 2023 release
  • Open App Api compliance August 2023 release

v1.8.0 - v.1.9.0

• Updated scheduler
• Updated reglog
• Gopherciser v0.17.4 -> v0.17.7
  • Changed parameters for prometheus push gateway
  • bugfixes

v1.7.0-v1.8.0

• Updated scheduler
• Gopherciser v0.17.0 -> v0.17.4
  • Configurable max websocket frame size
  • Improved disonnect discoverability
  • Add in-mem Get app structure method
  • [Apicompliance] openapp
  • Bugfixes

v1.6.0 - v1.7.0

• New installer - Option for personal install which should solve issues with "Administrator" privileges.
• Updated scheduler
• Gopherciser v0.16.2 -> v0.17.0
  • Bugfixes
  • Add option to print result json to script validate command
  • Updated api compliance for openhub action.

•  

•  

Your use of Qlik Sense Enterprise Scalability Tools will be subject to the same license agreement between you and Qlik for your Qlik Sense License. Qlik does not provide maintenance and support services for the Qlik Sense Enterprise Scalability Tools, however please check QlikCommunity for additional information on use of these products.

[Levi_Turner]

Looks like there's a doc bug (albeit minor).

In any of the examples where you use 

"claims": "{\"user\":\"{{.UserName}}}\",\"directory\":\"{{.Directory}}\"}"

This results in a user named user_#}

Like so:

 Which isn't exactly pretty 😀

Swap to:

"claims": "{\"user\":\"{{.UserName}}\",\"directory\":\"{{.Directory}}\"}"

[Daniel_Larsson]

Thank you Levi, I will make sure this is updated in the next release.

[sri_c003]

I am trying to get the app working. It says "connection successful". But when I run it I keep getting the error "Failed to perform app selection: app<Test App> not found in artifact map".

Is there some map where I need to add this app?

Command used:

gopherciser execute -c "path to script\Script1.json"

Attaching the script1.json to this post.

Any help would be appreciated.

[SebastianFabian]

Hi!

It's an easy fix. You just need to add an OpenHub action before your OpenApp action.

Alternatively, you can use the GUID instead of name to open the app directly.

The reason for this is the tool uses an internal "artifact map" that maps between the name of an app and a guid. When the OpenHub action occurs, the tool "sees" what apps are in the system and remembers them for future access. Then it uses this artifact map to look up the app. I attached a fixed version of your script.

[sri_c003]

That hit the spot, right on bulls eye! It started working as soon as i added "open hub". Thank you kind sir.

[sri_c003]

I updated the script to use generate users using a prefix, to scale to 25 users and for 25 iterations.

Created the scenario as below:

When the run the tool in command prompt, I am getting a couple of errors:

1. Sessions exceeded

2. Hub cannot be opened (presumably for hub was already open for the user from the first run) (maybe I am missing something that would allow to do "open hub" only in the first iteration and skip for the rest?

Most common error:
[90m11:26AM[0m [1m[31mERR[0m[0m failed to get DocList: EOF [36mAction=[0mopenhub [36mActionId=[0m25 [36mAppGUID=[0m [36mAppName=[0m [36mDetails=[0m [36mErrors=[0m0 [36mInfoType=[0m [36mLabel=[0m [36mObjectType=[0m [36mReceived=[0m0 [36mRequestsSent=[0m0 [36mResponseTime=[0m0 [36mSent=[0m0 [36mSession=[0m5 [36mSessionName=[0m [36mStack=[0m"EOF\nfailed to get DocList\ngithub.com/qlik-trial/gopherciser-oss/scenario.OpenHubSettings.Execute.func2\n\t/go/src/github.com/qlik-trial/gopherciser-oss/scenario/openhub.go:68\ngithub.com/qlik-trial/gopherciser-oss/session.(*PendingHandler).QueueRequest.func1\n\t/go/src/github.com/qlik-trial/gopherciser-oss/session/pendinghandler.go:88\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357" [36mThread=[0m1 [36mTick=[0m1767086597274 [36mUser=[0mQSEST_5 [36mWarnings=[0m0 [36mtimestamp=[0m2020-03-06T17:26:27.9188073Z

Any suggestions on these, please?

[SebastianFabian]

We seem to have a bug that sometimes causes the connection test to succeed even if the header settings are invalid. After double-checking the header name and header value against the QMC settings, the issue was resolved on my end. Would you please double check:

* Virtual proxy prefix

* Header name

* Header value

[sri_c003]

I had the previous test on November 2019 Patch 5. Now I am on February 2020.

I do not have an issue with connectivity, for it lists the apps on my test server properly. Attached is one run text file, and it has no errors.

The issue starts when it starts this in a loop for scalability testing. Below is the error when it starts iterating over 25 users over 25 runs.

ERR failed to get DocList: EOF Action=openhub ActionId=47 AppGUID= AppName= Details= Errors=0 InfoType= Label= ObjectType= Received=0 RequestsSent=0 ResponseTime=0 Sent=0 Session=16 SessionName= Stack="EOF\nfailed to get DocList\ngithub.com/qlik-trial/gopherciser-oss/scenario.OpenHubSettings.Execute.func2\n\t/go/src/github.com/qlik-trial/gopherciser-oss/scenario/openhub.go:68\ngithub.com/qlik-trial/gopherciser-oss/session.(*PendingHandler).QueueRequest.func1\n\t/go/src/github.com/qlik-trial/gopherciser-oss/session/pendinghandler.go:88\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357" Thread=2 Tick=4201200023 User=QSEST_16 Warnings=0 timestamp=2020-03-09T15:47:59.6543096Z
10:47AM ??? Action=openhub ActionId=47 AppGUID= AppName= Details=SenseEfW Errors=1 InfoType= Label= ObjectType= Received=132 RequestsSent=1 ResponseTime=0 Sent=84 Session=16 SessionName= Stack= Success=false Thread=2 Tick=4201200031 User=QSEST_16 Warnings=0 timestamp=2020-03-09T15:47:59.6543096Z

This error occurs pretty much at every run after the first.

And also the below one (was occurring in Nov 2019 P5 too).

10:50AM ??? {"jsonrpc":"2.0","method":"OnMaxParallelSessionsExceeded","params":{"severity":"fatal","timestamp":"2020-03-09T15:50:51.736728Z","message":"Access to the app is denied: MaxParallelSessionsExceeded"}} Action=openhub ActionId=497 AppGUID= AppName= Details=Received Errors=0 InfoType= Label= ObjectType= Received=0 RequestsSent=0 ResponseTime=0 Sent=0 Session=241 SessionName= Stack= Thread=15 Tick=4552568617 User=QSEST_241 Warnings=0 timestamp=2020-03-09T15:50:50.0148592Z

[SebastianFabian]

It sounds like you're encountering some issues related to your licenses. I note that you're using a list of 3 users that are invoked multiple times in parallel. Increasing the number of users in your user list to match the number of concurrent users in the script should mitigate your problems.

Refer to the following document:  Professional access allocations  

The maximum number of parallel user connections for a single user of this type of access pass is five (5).If you use a license with a signed license key, accessing the QMC also counts and adds to the maximum number of parallel sessions, which is five. To avoid unnecessary session consumption, the root admin should not be allocated any type of access.

and your test may also be affected by the "cool down" timer for licensed users, which is why it fails after the first time:

When a user with the maximum number of parallel user connections ends a connection (for example, by logging out) five minutes must pass before the user can use the access pass to add another connection (for example, by logging in).