Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
This forum is where any logged-in member can create a knowledge article.
A while ago, I conducted an internal research project with my colleagues @AlaneMiguelis, and Dr. @priscilarubim, aiming to validate the accuracy of a Two-Sample T-Test in Qlik by comparing it with the medical research gold standard: R. While we explored various hypothesis testing methods, we prioritized formalizing our findings on the T-Test. Recently, we decided to make this document available to the whole community.
The focus of our project was to assess whether the statistical results produced by Qlik matched those generated by R, particularly for the Two-Sample T-Test. Our methodology involved rigorous data analysis and repeated testing to ensure the reliability and consistency of our results. Through our detailed comparative analysis, we aimed to provide insights into the robustness of Qlik’s statistical capabilities. The code and dataset is available in a Git repository for those who want to reproduce the study.
Here is the abstract of our study. The full document can be accessed through the link at the end of this post.
ABSTRACT
Objective
The statistical determination of a large or small difference between two groups is not based on an absolute standard, but is rather an evaluation of the probability of an event.1,2 In the field of medical research, it is common to use statistical software for descriptive statistics as well as to perform statistical tests.3 However, most software provides ready-to-use functions, and the researchers have almost no information as to how those statistical tests are calculated inside those functions. This article evaluates the accuracy of two-sample Student’s t-test using Qlik analytics software. The gold standard used for this evaluation is the set of standard t-test functions available in R software, a widely used, robust, and reliable statistical software.5–7
Materials and Methods
The tests performed in this evaluation used a subset of Framingham heart study data. The dataset contains data on 4,434 anonymous participants, collected in three periods apart from each other by 6 years from 1956 to 1968. Five t-tests with 2 scenarios each were performed in Qlik analytics and in R and the results compared.
Results
In general, the results for multiple statistics obtained in Qlik analytics match the ones found in R for multiple scenarios: small and large sample sizes, small and large p-values, assuming and not assuming equal variance.
Discussion
Although Qlik analytics matches all statistics for t-test found in R, the p-value only matches up to four decimal points, which is concluded to be enough for testing hypothesis since the conventional levels of significance do not go lower than 0.1.
Conclusion
This research concluded that Qlik analytics can be used for two-sample t-tests in multiple scenarios.
Keywords: Qlik, t-test, r language, Framingham.
Link for the full article: Comparing Qlik Analytics and R Software for Accurate Two-Sample T-Tests Research Paper (ipc-global.com)
QlikView > Qlik Sense > Qlik Cloud...
Replatform is challenging and sometimes, different platforms have to coexist and interact to each other.
In this article I will show how can you call QlikView Tasks from Qlik Sense - and potentially from Qlik Cloud - using PowerShell Scripts.
# Open Powershell as Admin and Execute the following to allow Powershell Scripts execution
# Set-ExecutionPolicy -ExecutionPolicy RemoteSigned
# Set-ExecutionPolicy Unrestricted
The PowerShell script is simple as it shows below. It requires two parameters:
param (
[Parameter(Mandatory=$true)]
[string]$QlikViewQMCServiceUrl,
[Parameter(Mandatory=$true)]
[string]$TaskID
)
$service = New-WebServiceProxy -Uri $QlikViewQMCServiceUrl -Namespace QlikViewServer -UseDefaultCredential
$serviceKey = $service.GetTimeLimitedServiceKey()
$hdrs = @{}
$hdrs.Add("SOAPACTION","http://ws.qliktech.com/QMS/12/2/IQMS2/RunTask")
$hdrs.Add("Content-Type", "text/xml;charset=utf-8")
$hdrs.Add('X-Service-Key',$serviceKey)
$body = @{}
$body = '<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
<s:Body>
<RunTask xmlns="http://ws.qliktech.com/QMS/12/2/">
<taskID>' + $TaskID + '</taskID>
</RunTask>
</s:Body>
</s:Envelope>'
$res = Invoke-WebRequest -Uri $QlikViewQMCServiceUrl -Method Post -Body $body -UseDefaultCredential -Headers $hdrs
We can Run the PowerShell Script in multiple ways:
For now, we'll run it through Qlik Sense Load Script:
EXECUTE powershell.exe -ExecutionPolicy Bypass -File "$(vTempPowerShellScriptFile)" -QlikViewQMCServiceUrl "$(vTempQlikViewQMCServiceUrl)" -TaskID "$(vRunningTask)";
To call the EXECUTE statement in the Load Script you must enable the Legacy Mode in your Qlik Sense Server.
That is it! Now from Qlik Sense Server we can call QlikView Tasks with minimum changes in our server environments.
The PowerShell Script in my example contains only the core of its functionality. I strongly recommend including log messages and error handling through try/catch/finally blocks.
I haven't tested this but here is my theory:
That should do the magic. Please let me know if you have tried this and if it has worked.
Monitoring of Tasks that reload Apps in Qlik Sense.
One or more tasks are scheduled on the Qlik server that automatically load the Apps,
The goal is to be able to query the status of a task with the help of REST API technology and integrate it with the PRTG monitoring system.
Step 1 -> Creation of a dedicated Virtual Proxy in Qlik QMC.
The documentation to be able to proceed is available from the following link:
Step 2 -> API Query Mode and Test.
Once the JWT virtual proxy has been configured and the validity of the Token has been verified as per the official guide, it is possible to proceed with the desired query.
NB. In this case we validated the token using a Service user who also runs all the dedicated services including the Proxy Service.
In this case we will monitor the correct reload of the tasks on the QLIK server, the Tasks are scheduled on the server and allow the App data to be reloaded.
When one of these tasks fails via API call, it is possible to have a json result with the related details inside.
Below is an example of an API call:
This is an example of a task we want to monitor:
A quick method to test if the call was successful is from the browser using the Mode Header extension for google chrome described in step 1.
https://chrome.google.com/webstore/detail/modheader-modify-http-hea/idgpnmonknjnojddfkpgkljpfnnfcklj
So for the call to be successful you need to pass the following parameters:
Where Header name: Authorization and Header value: Bearer followed by the previously generated and tested Token and finally the Xrfkey key.
This will be the result in Json format returned by the browser if the call is successful:
Specifically in PRTG what we are going to monitor is:
"startTime": "2022-05-04T07:46:34.100Z"
"stopTime": "2022-05-04T07:46:37.636Z"
3. Other information that we may integrate and otherwise see in PRTG.
NB. A table with Qlik Sense reload task status codes and descriptions is available from the following link
https://help.qlik.com/en-US/alerting/July2023/Content/QlikAlerting/system-alerts.htm
This is an example of the string JSON with ok API call.
Step 3 -> Task monitoring from PRTG.
Type of sensor used in PRTG.
How to create a sensor.
The path on the PRTG server where the template is to be uploaded is as follows:
C:\Program Files (x86)\PRTG Network Monitor\Custom Sensors\rest
In the following path, however, it is necessary to define a Lookups in order to allow the sensor to interpret the various states of the Qlik Tasks listed in Step 2.
C:\Program Files (x86)\PRTG Network Monitor\lookups\custom
Result JSON query Task from Browser a part of this information will be used for the sensor in PRTG each task will correspond to a specific sensor.
The REST Custom Sensor was used in PRTG.
The REST Custom sensor queries a Representational State Transfer (REST) application programming interface (API) endpoint and maps the JavaScript Object Notation (JSON) or Extensible Markup Language (XML) result to the sensor values the mapping rule must be available as a REST configuration file in JSON template format (*.template) according to the PRTG API definition for custom sensors.
Two files have been created on the PRTG server.
Final integration flow monitoring Task PRTG -> QLIK.
How the DoD makes decisions is changing. Historical transaction reporting and static batch data analyses are giving way to predictive data science, streaming analytics, machine learning (ML), and artificial intelligence (AI). The DoD must embrace these new paradigms for more informed, real-time decisions to impact business, operations, and mission performance.
This whitepaper will address how Qlik can enable the DoD to achieve it's mission.