Skip to main content
Announcements
Qlik Community Office Hours, March 20th. Former Talend Community users, ask your questions live. SIGN UP
cancel
Showing results for 
Search instead for 
Did you mean: 
Frederic_De_Ranter

[deprecated] Qlik Hardware Benchmarking Package (Qlik Sense and QlikView)

This package is deprecated and will no longer be maintained.

The document attached outlines all required information on how to access/execute/analyze results for the Qlik Hardware Benchmarking test package.

The goal with the HW Benchmarking package is to have a unified Hardware test that can be executed for various environments with the aim to establish a matrix for comparisons on how different servers and configurations perform with QlikView and/or Qlik Sense. This package can be used for analytics on your own to perform comparisons of performance for different hardware configurations.

The document contains information on how to download additional files needed. The package supports Qlik Sense Enterprise on Windows and QlikView Server 12 or newer.

This package is deprecated and will no longer be maintained.

14 Replies
scullion
Contributor
Contributor

Hi, either the PDF or the package is incorrect. the pdf makes reference to some powershell scripts but in the package there are just .bat. the pdf is dated sep 2020 and the packages last update in 2018 so maybe the new pachages have not been uploaded ? 

Frederic_De_Ranter
Author

Hi, 

There was indeed an issue with the shared folder. Now it should be corrected and the new HW BM package should be available in the "HW BM package" folder.

Best regards,
Frederic

Oleg_Troyansky
Partner Ambassador/MVP
Partner Ambassador/MVP

This is great, should be very useful indeed! Does it support local Desktop versions of QS and QV - to help developers benchmark their laptops for Qlik performance?

Thank you!

Frederic_De_Ranter
Author

Hi Oleg,

Only QlikView Server and Qlik Sense Server for Windows are supported. The application and load are designed for larger systems.

Best regards,
Frederic

eyalnir_qlik
Partner - Creator
Partner - Creator

Hello, we encountered with issue refer to  CPU's Benchmarking Selection Performance, new server Intel (Xeon Gold 5220R 2.2 GHz) order for sense server , the main issue was that adding 48 cores or 24 cores cause performance decrease Vs. only 4 or 8 cores , enclosed his benchmark report, i've also test the same scenario with Intel Xeon Platinum 8168 Processor and i get the same performance decrease, we need to understand why sense engine can't use more then 12-16 cores for consumer server, it's seem like a defect, appreciate your feedback regarding the Benchmark Test

Test

Sense Version

Memory RAM

Number of Cores Intel (Xeon Gold 5220R 2.2 GHz)

Time to Selection (Sec)

1

Ver. April 2020

1.5TB 3200Mhz (24 Slots)

48 Cores 96 Logical Processors

~3 Sec

2

Ver. February 2021

1.5TB 3200Mhz (24 Slots)

48 Cores 96 Logical Processors

~6 Sec

3

Ver. April 2020

1.5TB 3200Mhz (24 Slots)

8 Cores 8 Logical Processors

Less than1 Sec

4

Ver. February 2021

1.5TB 3200Mhz (24 Slots)

4 Cores 4 Logical Processors

Less than1 Sec

Thanks !

 

Frederic_De_Ranter
Author

Hi Eyal,

You're not using the benchmarking package that we provide above so it is very hard for us to provide any feedback in why your app is performing the way it does. The Qlik engine will scale performance with the amount of cores (we have tested it up to 256 logical cores) when the app is well designed. 

A possible explanation for the behaviour you see, might be that the clock turbo boost speed for your CPU (the gold 5220R) is rather high for when few cores are used (1 core can go up to 4GHz) but this goes down the more cores are active in the CPU. It would indicate that the data model/calculations in the app you are using are not benefiting from having more cores available. 

Best regards,
Frederic

eyalnir_qlik
Partner - Creator
Partner - Creator

Hi Frederic

Thanks for your feedback and elaborate explanation, 

Note that in this case, customer Production Server has no clock turbo boost speed for CPU and the same issue appears, also "app is well designed" isn't relevant in our scenario, since we got better performance with less cores, hence we suspect that there is a defect with sense engine 

Thanks

Eyal 

kss7374
Partner - Contributor III
Partner - Contributor III

Frederick, have a good time!

I ran into a problem when using the HWBM-Package-1.0 toolkit.. It gives an error when running 4 tests (Preload-preloadhicached-Template and Test4a-lothic-Template, Test4b-hitic-Template, Test4c-userpush-Template), all the remaining tests work fine and I analyze them in QlikView.
Testing:
appl "200M-ptsalesanalytics-v3" (October 15 2020, 5.20Gb) in QS version 14.28sr3;
$ToolFolder = "C:\Distr\4_QlikSenseEnterpriseScalabilityTools\QSEST-1.2.0-windows\gopherciser";
Package: HWBM-Package-1.0.zip (October 15 2020, 2.48Mb)

Errors in the log are the same for all phases (PreloadHiCached, Test4a, Test4b, Test4c )

################################## repo is running on 40 cores
===================Preload-PreloadHiCached - qs01 ========================
running test for gopher
Version: v0.14.0
JSONParseError: Failed to unmarshal config from json: unmarshalerDecoder: Failed unmarshaling config: config.cfgCore.Scenario: []scenario.Action: unmarshalerDecoder: Invalid action<StaticSelect>, error found in #10 byte of ...|
}
},
|..., bigger context ...|e,
"rows": [1,2],
"cols": [0]
}
},
{
"label": "Table Sales by Department |..., error found in #10 byte of ...| }
]
}
|..., bigger context ...|
"label": "Clear all selections 4"
}
]
}
|...
It is impossible to call a method for an expression with a NULL value.
C:\HWBMPackage\Execute-HWBM_40_128.ps1:736 sign:13
+ $testOutputArray = $testOutput.Split('<>')
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull

Cannot index to NULL array.
C:\HWBMPackage\Execute-HWBM_40_128.ps1:737 sign:13
+ $script:tempErrors = [int]$testOutputArray[-10]
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : NullArray

 

Thanks

Best regards,
Sergey

Frederic_De_Ranter
Author

Hi Sergey,

Yes, there has been a change in the latest gopherciser that breaks these tests. I will come with a fix for the HWBM package.

Best regards,
Frederic