topic Re: Qlik Hardware Benchmarking Package (Qlik Sense and QlikView) in Scalability

[deprecated] Qlik Hardware Benchmarking Package (Qlik Sense and QlikView)

Frederic_De_Ranter — Thu, 18 Aug 2022 11:51:18 GMT

This package is deprecated and will no longer be maintained.

The document attached outlines all required information on how to access/execute/analyze results for the Qlik Hardware Benchmarking test package.

The goal with the HW Benchmarking package is to have a unified Hardware test that can be executed for various environments with the aim to establish a matrix for comparisons on how different servers and configurations perform with QlikView and/or Qlik Sense. This package can be used for analytics on your own to perform comparisons of performance for different hardware configurations.

The document contains information on how to download additional files needed. The package supports Qlik Sense Enterprise on Windows and QlikView Server 12 or newer.

This package is deprecated and will no longer be maintained.

Re: Qlik Hardware Benchmarking Package (Qlik Sense and QlikView)

scullion — Thu, 15 Oct 2020 13:39:22 GMT

Hi, either the PDF or the package is incorrect. the pdf makes reference to some powershell scripts but in the package there are just .bat. the pdf is dated sep 2020 and the packages last update in 2018 so maybe the new pachages have not been uploaded ?

Re: Qlik Hardware Benchmarking Package (Qlik Sense and QlikView)

Frederic_De_Ranter — Thu, 15 Oct 2020 16:02:08 GMT

Hi,

There was indeed an issue with the shared folder. Now it should be corrected and the new HW BM package should be available in the "HW BM package" folder.

Best regards,
Frederic

Re: Qlik Hardware Benchmarking Package (Qlik Sense and QlikView)

Oleg_Troyansky — Sat, 17 Oct 2020 22:57:22 GMT

This is great, should be very useful indeed! Does it support local Desktop versions of QS and QV - to help developers benchmark their laptops for Qlik performance?

Thank you!

Re: Qlik Hardware Benchmarking Package (Qlik Sense and QlikView)

Frederic_De_Ranter — Mon, 19 Oct 2020 13:58:07 GMT

Hi Oleg,

Only QlikView Server and Qlik Sense Server for Windows are supported. The application and load are designed for larger systems.

Best regards,
Frederic

Re: Qlik Hardware Benchmarking Package (Qlik Sense and QlikView)

eyalnir_qlik — Thu, 13 May 2021 09:52:07 GMT

Hello, we encountered with issue refer to CPU's Benchmarking Selection Performance, new server Intel (Xeon Gold 5220R 2.2 GHz) order for sense server , the main issue was that adding 48 cores or 24 cores cause performance decrease Vs. only 4 or 8 cores , enclosed his benchmark report, i've also test the same scenario with Intel Xeon Platinum 8168 Processor and i get the same performance decrease, we need to understand why sense engine can't use more then 12-16 cores for consumer server, it's seem like a defect, appreciate your feedback regarding the Benchmark Test

Test	Sense Version	Memory RAM	Number of Cores Intel (Xeon Gold 5220R 2.2 GHz)	Time to Selection (Sec)
1	Ver. April 2020	1.5TB 3200Mhz (24 Slots)	48 Cores 96 Logical Processors	~3 Sec
2	Ver. February 2021	1.5TB 3200Mhz (24 Slots)	48 Cores 96 Logical Processors	~6 Sec
3	Ver. April 2020	1.5TB 3200Mhz (24 Slots)	8 Cores 8 Logical Processors	Less than1 Sec
4	Ver. February 2021	1.5TB 3200Mhz (24 Slots)	4 Cores 4 Logical Processors	Less than1 Sec

Thanks !

Re: Qlik Hardware Benchmarking Package (Qlik Sense and QlikView)

Frederic_De_Ranter — Tue, 25 May 2021 09:17:02 GMT

Hi Eyal,

You're not using the benchmarking package that we provide above so it is very hard for us to provide any feedback in why your app is performing the way it does. The Qlik engine will scale performance with the amount of cores (we have tested it up to 256 logical cores) when the app is well designed.

A possible explanation for the behaviour you see, might be that the clock turbo boost speed for your CPU (the gold 5220R) is rather high for when few cores are used (1 core can go up to 4GHz) but this goes down the more cores are active in the CPU. It would indicate that the data model/calculations in the app you are using are not benefiting from having more cores available.

Best regards,
Frederic

Re: Qlik Hardware Benchmarking Package (Qlik Sense and QlikView)

eyalnir_qlik — Thu, 03 Jun 2021 07:02:22 GMT

Hi Frederic

Thanks for your feedback and elaborate explanation,

Note that in this case, customer Production Server has no clock turbo boost speed for CPU and the same issue appears, also "app is well designed" isn't relevant in our scenario, since we got better performance with less cores, hence we suspect that there is a defect with sense engine

Thanks

Eyal

Re: Qlik Hardware Benchmarking Package (Qlik Sense and QlikView)

kss7374 — Tue, 09 Nov 2021 09:35:17 GMT

Frederick, have a good time!

I ran into a problem when using the HWBM-Package-1.0 toolkit.. It gives an error when running 4 tests (Preload-preloadhicached-Template and Test4a-lothic-Template, Test4b-hitic-Template, Test4c-userpush-Template), all the remaining tests work fine and I analyze them in QlikView.
Testing:
appl "200M-ptsalesanalytics-v3" (October 15 2020, 5.20Gb) in QS version 14.28sr3;
$ToolFolder = "C:\Distr\4_QlikSenseEnterpriseScalabilityTools\QSEST-1.2.0-windows\gopherciser";
Package: HWBM-Package-1.0.zip (October 15 2020, 2.48Mb)

Errors in the log are the same for all phases (PreloadHiCached, Test4a, Test4b, Test4c )

################################## repo is running on 40 cores
===================Preload-PreloadHiCached - qs01 ========================
running test for gopher
Version: v0.14.0
JSONParseError: Failed to unmarshal config from json: unmarshalerDecoder: Failed unmarshaling config: config.cfgCore.Scenario: []scenario.Action: unmarshalerDecoder: Invalid action<StaticSelect>, error found in #10 byte of ...|
}
},
|..., bigger context ...|e,
"rows": [1,2],
"cols": [0]
}
},
{
"label": "Table Sales by Department |..., error found in #10 byte of ...| }
]
}
|..., bigger context ...|
"label": "Clear all selections 4"
}
]
}
|...
It is impossible to call a method for an expression with a NULL value.
C:\HWBMPackage\Execute-HWBM_40_128.ps1:736 sign:13
+ $testOutputArray = $testOutput.Split('<>')
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull

Cannot index to NULL array.
C:\HWBMPackage\Execute-HWBM_40_128.ps1:737 sign:13
+ $script:tempErrors = [int]$testOutputArray[-10]
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : NullArray

Thanks

Best regards,
Sergey

Re: Qlik Hardware Benchmarking Package (Qlik Sense and QlikView)

Frederic_De_Ranter — Fri, 12 Nov 2021 10:16:18 GMT

Hi Sergey,

Yes, there has been a change in the latest gopherciser that breaks these tests. I will come with a fix for the HWBM package.

Best regards,
Frederic

Re: Qlik Hardware Benchmarking Package (Qlik Sense and QlikView)

Frederic_De_Ranter — Tue, 23 Nov 2021 10:33:00 GMT

Hi Sergey,

The HW BM package is now updated to fix the above tests.

Best regards,
Frederic

Re: Qlik Hardware Benchmarking Package (Qlik Sense and QlikView)

kss7374 — Wed, 24 Nov 2021 14:19:02 GMT

Hello, Frederic
Downloaded the updated package (v1.1), but now the problem is at the very beginning.
The client connects to the server, restarts the services, but does not go further, hangs on checking IIS (Qlik Sense Services http check Waiting for all the start-up tasks to be done... (if this keeps repeating: check if IIS is running on port 80 on the proxyServer)).
If I run the script from the old package, it normally restarts the services and starts running tests.

I started comparing the code, there are a lot of changes, there are differences, but perhaps you can tell me what the problem is?

I tried to specify the short and full domain name of the server - but the result is the same...

PS C:\HWBM-Package> $PSVersionTable.PSVersion

Major Minor Build Revision
----- ----- ----- --------
5 1 17763 2183

PS C:\HWBM-Package> Get-ExecutionPolicy
RemoteSigned
PS C:\HWBM-Package> $mProxyHostname = "qs01"
PS C:\HWBM-Package> Invoke-WebRequest -Uri http://$mProxyHostname/hub

StatusCode : 200
StatusDescription : OK
Content : <!doctype html>
<html lang="en">
<head>
<meta name="robots" content="noindex, nofollow" />
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
<title>Qlik Sense login page<...
RawContent : HTTP/1.1 200 OK
Content-Length: 97760
Cache-Control: no-cache, no-store
Content-Type: text/html
Date: Wed, 24 Nov 2021 14:09:37 GMT
Server: Microsoft-HTTPAPI/2.0

<!doctype html>
<html lang="e...
Forms : {login-form}
Headers : {[Content-Length, 97760], [Cache-Control, no-cache, no-store], [Content-Type, text/html], [Date, Wed, 24 Nov 2021 14:09:37 GMT]...}
Images : {@{innerHTML=; innerText=; outerHTML=<img alt="Qlik Sense" src="data&colon;image/png;base64,

..... .... .....

Links : {}
ParsedHtml : mshtml.HTMLDocumentClass
RawContentLength : 97760

Re: Qlik Hardware Benchmarking Package (Qlik Sense and QlikView)

Frederic_De_Ranter — Wed, 24 Nov 2021 19:12:59 GMT

Hi,

The script expects that you can not get this page since you are not logged in ("Unauthorized"), if you have configured your server differently this may not work.

The previous version did the same but with a different check since the error message has changed with the latest version of powershell. Changing the Uri to point to the header proxy might be a solution.

Invoke-WebRequest -Uri http://$mProxyHostname/header/hub

Else you can change

$HostisDown = 1

Start-Sleep 20

$HostisDown = 0

Then the check doesn't run but it does wait 20 seconds extra to give some time for the host to come up.

BR,

Frederic

Re: Qlik Hardware Benchmarking Package (Qlik Sense and QlikView)

kss7374 — Thu, 25 Nov 2021 07:24:33 GMT

Hello, Frederic

Thank you for your help - it works.

I have PowerShell version 5.X running with this condition :

Measure-Command { Invoke-WebRequest -Uri http://$fullProxyHostname/header/hub} >$null 2>&1
if ($error[0].Exception -like '*(401)*') {$HostisDown=0}

But it does not work with such a condition (by default):

Measure-Command { Invoke-WebRequest -Uri http://$fullProxyHostname/hub/} >$null 2>&1
if ($error[0].Exception -like '*Unauthorized*') {$HostisDown=0}

Perhaps in the documentation for the package you need to make a comment or check for the version of PowerShell you are using?

BR,

Sergey

Re: Qlik Hardware Benchmarking Package (Qlik Sense and QlikView)

kss7374 — Thu, 25 Nov 2021 11:23:43 GMT

Hello, Frederic!

Tests from the updated package "HWBM-Package-1.1" they work fine for QlikSense, but there are several types of errors that occur in tests (3High Load Test, Test4a-Lothic, Test4b-hitic, Test4c-userpush, Test6-cpucapacity), for which I could not find information in the Community and in the documentation:

* Unexpected status code: 500 ProxyErrorCode.1000 (get http://qs01:80/header/api/hub/v1/properties) unexpected response status code<500> expected<[200]>
* Failed connecting to sense server: Failed connecting to sense server: websocket connected, but received error topic/-s: OnLicenseAccessDenied
* Failed connecting to sense server: Failed connecting to sense server: websocket connected, but no state created

Please tell me what they are caused by, how they can be diagnosed and corrected:
- the load on the server at the time of the test?
- is this a license restriction?
(I noticed that during the execution of all tests, more than 3.5 thousand temporary gopher_nnnn users are created in the system)

Thanks,

Sergey