Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Search our knowledge base, curated by global Support, for answers ranging from account questions to troubleshooting error messages.
Qlik offers a wide range of channels to assist you in troubleshooting, answering frequently asked questions, and getting in touch with our technical experts. In this article, we guide you through all available avenues to secure your best possible experience.
For details on our terms and conditions, review the Qlik Support Policy.
Index:
We're happy to help! Here's a breakdown of resources for each type of need.
Support | Professional Services (*) | |
Reactively fixes technical issues as well as answers narrowly defined specific questions. Handles administrative issues to keep the product up-to-date and functioning. | Proactively accelerates projects, reduces risk, and achieves optimal configurations. Delivers expert help for training, planning, implementation, and performance improvement. | |
|
|
(*) reach out to your Account Manager or Customer Success Manager
Your first line of support: https://community.qlik.com/
Looking for content? Type your question into our global search bar:
Leverage the enhanced and continuously updated Knowledge Base to find solutions to your questions and best practice guides. Bookmark this page for quick access!
Subscribe to maximize your Qlik experience!
The Support Updates Blog
The Support Updates blog delivers important and useful Qlik Support information about end-of-product support, new service releases, and general support topics. (click)
The Qlik Design Blog
The Design blog is all about product and Qlik solutions, such as scripting, data modelling, visual design, extensions, best practices, and more! (click)
The Product Innovation Blog
By reading the Product Innovation blog, you will learn about what's new across all of the products in our growing Qlik product portfolio. (click)
Q&A with Qlik
Live sessions with Qlik Experts in which we focus on your questions.
Techspert Talks
Techspert Talks is a free webinar to facilitate knowledge sharing held on a monthly basis.
Technical Adoption Workshops
Our in depth, hands-on workshops allow new Qlik Cloud Admins to build alongside Qlik Experts.
Qlik Fix
Qlik Fix is a series of short video with helpful solutions for Qlik customers and partners.
Suggest an idea, and influence the next generation of Qlik features!
Search & Submit Ideas
Ideation Guidelines
Get the full value of the community.
Register a Qlik ID:
Incidents are supported through our Chat, by clicking Chat Now on any Support Page across Qlik Community.
To raise a new issue, all you need to do is chat with us. With this, we can:
Log in to manage and track your active cases in Manage Cases. (click)
Please note: to create a new case, it is easiest to do so via our chat (see above). Our chat will log your case through a series of guided intake questions.
When creating a case, you will be prompted to enter problem type and issue level. Definitions shared below:
Select Account Related for issues with your account, licenses, downloads, or payment.
Select Product Related for technical issues with Qlik products and platforms.
If your issue is account related, you will be asked to select a Priority level:
Select Medium/Low if the system is accessible, but there are some functional limitations that are not critical in the daily operation.
Select High if there are significant impacts on normal work or performance.
Select Urgent if there are major impacts on business-critical work or performance.
If your issue is product related, you will be asked to select a Severity level:
Severity 1: Qlik production software is down or not available, but not because of scheduled maintenance and/or upgrades.
Severity 2: Major functionality is not working in accordance with the technical specifications in documentation or significant performance degradation is experienced so that critical business operations cannot be performed.
Severity 3: Any error that is not Severity 1 Error or Severity 2 Issue. For more information, visit our Qlik Support Policy.
If you require a support case escalation, you have two options:
A collection of useful links.
Qlik Cloud Status Page
Keep up to date with Qlik Cloud's status.
Support Policy
Review our Service Level Agreements and License Agreements.
Live Chat and Case Portal
Your one stop to contact us.
By default, API keys have either 6 hours or a day-long expiration day.
You can define your own maximum expiration for your tenant.
Go to the Management Console, select Settings> API keys, find Change Maximum token expiration and set the new maximum duration in days.
For more information, see Enabling API keys in the tenant.
The information in this article is provided as-is and to be used at own discretion. Depending on tool(s) used, customization(s), and/or other factors ongoing support on the solution below may not be provided by Qlik Support.
Qlik Fix: How to Generate API Keys with APIs in Qlik Sense Enterprise SaaS
Obtaining API keys for managing instances in SaaS editions of Qlik Sense
Qlik Sense Enterprise on Windows is reliant on the availability and performance of its server cluster share, where the Qlik Sense apps are stored.
In some instances, the file storage maybe not be compatible with the requirements of Qlik Sense, and this can lead to app corruption. An example of this may be a Distributed File System (DFS) managed share with replication enabled, which can have side effects such as app corruption, apps being removed from disk, and/or not correctly replicated.
In a Qlik Sense Enterprise on Windows site, a file share is used to store the binary app data including the data models and dashboard sheets. It can be located on one of the nodes in the Sense site or located on a dedicated file server for better resilience and performance.
Note: Each Qlik Sense Enterprise on Windows site requires its own file share with its own set of binaries, archives, extensions, etc. If two mirrored environments are configured to use the same files in only one share, besides potentially experiencing file locking issues, the Qlik Sense database will not have the correct references (metadata) pertaining to the information stored in the share. So you will have two Qlik Sense environments updating files on the shares and which are not aware of updates performed by the other environment(s). Performing such action is highly unsupported and will cause malfunction.
See Qlik Sense for Administrators: Persistence for more details on requirements for a file share.
In the case of DFS managed shares, the replication feature on file level can lead to inconsistency with the Qlik Sense apps stored on disk.
Provided that the requirements detailed in this document are met, Qlik Sense will not know the difference between one and the other, but choosing the wrong storage solution can result in bad performance and instability that can be very difficult to troubleshoot.
When your storage is on-premise, Qlik recommends relying on enterprise SAN solutions that can be managed and configured according to your needs. A NAS is better suited for serving files to client machines, while a SAN can be relied upon by multiple servers to give adequate throughput and latency to large blocks of storage. These are typically set up with a fiber channel backbone that can sustain the demand of a large and busy Qlik Sense environment.
For Virtual Private Cloud (VPC) deployments, be aware of multiple instance types and what type of workloads they are suitable for. A Windows-based File Server will run better in a Storage-Optimised virtual machine than a general purpose instance. Consider using block storage services instead of Windows-based, to take advantage of scaling and resilience offered by those providers.
Performance: For any network storage resource, the network backbone throughput and latency will directly influence performance of all file-based operations performed by the Qlik Sense platform on the configured file shares. Examples: first-time loading of apps from file share into Engine memory; saving and refreshing of app changes; log archiving, etc.
For Windows-hosted file shares, the sizing of the File Server machine will directly influence performance of those file shares. Using Administrative Shares instead of UNC paths can severely impact performance, upwards of 50% and causing other issues, such as File Locking.
BAD path example: \\FILESERVER\C$\QlikSenseServiceCluster
GOOD path examples: \\FILESERVER\QlikSenseServiceCluster or \\FILESERVER\QlikDataFiles
It is recommended to remove / disable Administrative shares to prevent unauthorised access to resource and potentially avoid configuration issues between these shares and Qlik Client Managed products.
Qlik recommends a maximum latency of 4 ms between Qlik Sense servers and any file shares. 10 Gbit networking is recommended, considering many parallel file read/write operations can be requested from multiple file share clients (Qlik Sense servers).
Note: Qlik cannot verify support for all storage vendors and Qlik recommends that customers test their preferred infrastructure. In the event of an issue arising that is attributed to storage, Qlik Support may request that customers attempt to replicate the issue on a Windows hosted file share.
Protocol support:
Reload fails in QMC even though script part is successfull in Qlik Sense Enterprise on Windows November 2023 and above.
When you are using a NetApp based storage you might see an error when trying to publish and replace or reloading a published app.
In the QMC you will see that the script load itself finished successfully, but the task failed after that.
ERROR QlikServer1 System.Engine.Engine 228 43384f67-ce24-47b1-8d12-810fca589657
Domain\serviceuser QF: CopyRename exception:
Rename from \\fileserver\share\Apps\e8d5b2d8-cf7d-4406-903e-a249528b160c.new
to \\fileserver\share\Apps\ae763791-8131-4118-b8df-35650f29e6f6
failed: RenameFile failed in CopyRename
ExtendedException: Type '9010' thrown in file
'C:\Jws\engine-common-ws\src\ServerPlugin\Plugins\PluginApiSupport\PluginHelpers.cpp'
in function 'ServerPlugin::PluginHelpers::ConvertAndThrow'
on line '149'. Message: 'Unknown error' and additional debug info:
'Could not replace collection
\\fileserver\share\Apps\8fa5536b-f45f-4262-842a-884936cf119c] with
[\\fileserver\share\Apps\Transactions\Qlikserver1\829A26D1-49D2-413B-AFB1-739261AA1A5E],
(genericException)'
<<< {"jsonrpc":"2.0","id":1578431,"error":{"code":9010,"parameter":
"Object move failed.","message":"Unknown error"}}
ERROR Qlikserver1 06c3ab76-226a-4e25-990f-6655a965c8f3
20240218T040613.891-0500 12.1581.19.0
Command=Doc::DoSave;Result=9010;ResultText=Error: Unknown error
0 0 298317 INTERNAL&
emsp; sa_scheduler b3712cae-ff20-4443-b15b-c3e4d33ec7b4
9c1f1450-3341-4deb-bc9b-92bf9b6861cf Taskname Engine Not available
Doc::DoSave Doc::DoSave 9010 Object move failed.
06c3ab76-226a-4e25-990f-6655a965c8f3
Potential workarounds
The most plausible cause currently is that the specific engine version has issues releasing File Lock operations. We are actively investigating the root cause, but there is no fix available yet.
An update will be provided as soon as there is more information to share.
QB-25096
Qlik Sense can be configured to allow both HTTP and HTTPS traffic. In some scenarios, an administrator prefers to only enable HTTPS traffic and redirect users who attempt to access Qlik Sense by HTTP to HTTPS.
Most web servers like IIS, Apache, or nginx set up as reverse proxies in front of Qlik Sense can accomplish a redirect from HTTP to HTTPS. Accomplishing the redirect from HTTP to HTTPS is the preferred method. The code below is provided as is, and is not guaranteed to continue working in case Qlik upgrades the node.js version used in the Qlik Sense base product.
It is recommended to use whichever product the organization has institutional knowledge about since support will need to be provided from those resources.
For demonstration purposes in the Qlik Sense product, the administrator can use a custom NodeJS module which is launched by Qlik Sense to accomplish this task.
! Qlik Support's scope does not include the debugging or writing of custom code for customers. For assistance on custom solutions, please contact our consulting services/professional services, or engage with our active community of partners and customers. The supplied example code is provided for demonstration purposes. No Support or maintenance is implied or provided. Further customization is expected to be necessary and it is the responsibility of the end administrator to test and implement an appropriate rule for their specific use case.
! Redirecting from HTTP to HTTPS does not automatically guarantee or equal HSTS compliance. For details on HSTS compliance, see HTTP Strict Transport Security (HSTS) in Qlik Sense.
! Changes made to the configuration files will be reverted back to default upon product upgrade.
1. Create a folder named route2https in C:\Program Files\Qlik\Sense\ServiceDispatcher\Node\
2. Create server.js file in the route2https folder, paste this code in to it and save:
// Redirect from http port 80 to https var http = require('http'); http.createServer(function (req, res) { res.writeHead(301, { "Location": "https://" + req.headers['host'] + req.url }); res.end(); }).listen(80);
3. Edit C:\Program Files\Qlik\Sense\ServiceDispatcher\services.conf file and paste the following code into it at the end:
[httproute] Identity=Qlik.httproute Enabled=true DisplayName=httproute ExecType=nodejs ExePath=Node\node.exe Script=Node\route2https\server.js
4. Restart the Qlik Sense Service Dispatcher service
These steps need to be carried out on each Proxy node that is expected to redirect from HTTP to HTTPS.
* Courtesy of Nick Akincilar in Qlik Pre-Sales.
For an additional example see the Qlik Community: Qlik Sense redirect HTTP to HTTPS
What versions of PostgreSQL does Qlik Sense support?
All up-to-date information regarding PostgreSQL compatibility with supported Qlik Sense Enterprise on Windows versions can be found in the System Requirements of the respective release.
This article only documents older and out-of-support Qlik Sense versions.
Sense May 2021 is compatible with PostgreSQL 9.6.x, 11.x and 12.5 (included)
Sense February 2021 PostgreSQL 9.6.x (included in the installer), 11.x (not included in the installer), 12.x (not included in the installer, must not be used with Centralized logging )
Sense February to November 2020 is compatible with PostgreSQL 9.6.x (included), 11.x
Sense November 2019 is compatible with PostgreSQL 9.6.x (included), 11.x
Sense September 2017 is compatible with PostgreSQL version 9.6.x
Sense June 2017 is compatible with PostgreSQL version 9.6.x
Sense 3.2 is compatible with PostgreSQL version 9.3.x and 9.6.x
Sense 3.1 is compatible with PostgreSQL version 9.3.x
This method only applies to bundled installs. If you have installed PostgreSQL separately, verify the version using for example the installed programs in Windows.
Installing, upgrading, and managing the Qlik Cloud Monitoring Apps has just gotten a whole lot easier! With two new Qlik Application Automation templates coupled with Qlik Data Alerts, you can now:
The above allows you to deploy the monitoring apps to your tenant with a hands-off approach. Dive into the individual components below.
Content:
This automation template is a fully guided installer/updater for the Qlik Cloud Monitoring Applications, including but not limited to the App Analyzer, Entitlement Analyzer, Reload Analyzer, and Access Evaluator applications. Leverage this automation template to quickly and easily install and update these or a subset of these applications with all their dependencies. The applications themselves are community-supported; and, they are provided through Qlik's Open-Source Software GitHub and thus are subject to Qlik's open-source guidelines and policies.
For more information, refer to the GitHub repository.
Note that if the monitoring applications have been installed manually (i.e., not through this automation) then they will not be detected as existing. The automation will install new copies side-by-side. Any subsequent executions of the automation will detect the newly installed monitoring applications and check their versions, etc. This is due to the fact that the applications are tagged with "QCMA - {appName}" and "QCMA - {version}" during the installation process through the automation. Manually installed applications will not have these tags and therefore will not be detected.
This template is intended to be used alongside the Qlik Cloud Monitoring Apps for user-based subscriptions template. This automation provides the ability to keep the API key and associated data connection used for the Qlik Cloud Monitoring Apps up to date on a scheduled basis. Simply input the space Id where the monitoring_apps_REST data connection should reside, and the automation will recreate both the API key and data connection regularly. Ensure that the cadence of the automation’s schedule is less than the expiry of the API key.
Enter in the Id of the space where the monitoring_apps_REST data connection should reside.
Ensure that this automation is run off-hours from your scheduled monitoring application reloads so it does not disrupt the reload process.
Each Qlik Cloud Monitoring App has the following two variables:
With these variables, we can create a new Qlik Data Alert on a per-app basis. For each monitoring app that you want to be notified on if it falls out of date:
Here is an example of an alert received for the App Analyzer, showing that at this point in time, the latest version of the application is 5.1.3 and that the app is out of date:
Q: Can I re-run the installer to check if any of the monitoring applications are able to be upgraded to a later version?
A: Yes. Run the installer, select which applications should be checked and select the space that they reside in. If any of the selected applications are not installed or are upgradeable, a prompt will appear to continue to install/upgrade for the relevant applications.
Q: What if multiple people install monitoring applications in different spaces?
A: The template scopes the applications install process to a “target” space, i.e., a shared space (if not published) or a managed space. It will scope the API key name to `QCMA – {spaceId}` of that target space. This allows the template to install/update the monitoring applications across spaces and across users. If one user installs an application to “Space A” and then another user installs a different monitoring application to “Space A”, the template will see that a data connection and associated API key (in this case from another user) exists for that space already and it will install the application leveraging those pre-existing assets.
Q: What if a new monitoring application is released? Will the template provide the ability to install that application as well?
A: Yes. The template receives the list of applications dynamically from GitHub. If a new monitoring application is released, it will become available immediately through the template.
Q: I would like to be notified whenever a new version of a monitoring applications is released. Can this template do that?
A: As per the article above, the automation templates are not responsible for notifications of whether the applications are out of date. This is achieved using Qlik Alerting on a per-application basis as described in Part 3.
Q:I have updated my application, but I noticed that it did not preserve the history. Why is that?
A: The history is preserved in the prior versions of the application’s QVDs so the data is never deleted and can be loaded into the older version. Each upgrade will generate a new set of QVDs as the data models for the applications sometimes change due to bug fixes, updates, new features, etc. If you want to preserve the history when updating, the application can be upgraded with the “Publish side-by-side” method so that the older version of the application will remain as an archival application. However note that the Qlik Alert (from Part 3) will need to be recreated and any community content that was created on the older application will not be transferred to the new application.
This Techspert Talks session addresses:
Tip: Download the LogAnalyzer app here: LogAnalysis App: The Qlik Sense app for troubleshooting Qlik Sense Enterprise on Windows logs.
00:00 - Intro
01:22 - Multi-Node Architecture Overview
04:10 - Common Performance Bottlenecks
05:38 - Using iPerf to measure connectivity
09:58 - Performance Monitor Article
10:30 - Setting up Performance Monitor
12:17 - Using Relog to visualize Performance
13:33 - Quick look at Grafana
14:45 - Qlik Scalability Tools
15:23 - Setting up a new scenario
18:26 - Look QSST Analyzer App
19:21 - Optimizing the Repository Service
21:38 - Adjusting the Page File
22:08 - The Sense Admin Playbook
23:10 - Optimizing PostgreSQL
24:29 - Log File Analyzer
27:06 - Summary
27:40 - Q&A: How to evaluate an application?
28:30 - Q&A: How to fix engine performance?
29:25 - Q&A: What about PostgreSQL 9.6 EOL?
30:07 - Q&A: Troubleshooting performance on Azure
31:22 - Q&A: Which nodes consume the most resources?
31:57 - Q&A: How to avoid working set breaches on engine nodes?
34:03 - Q&A: What do QRS log messages mean?
35:45 - Q&A: What about QlikView performance?
36:22 - Closing
Resources:
LogAnalysis App: The Qlik Sense app for troubleshooting Qlik Sense Enterprise on Windows logs
Qlik Help – Deployment examples
Using Windows Performance Monitor
PostgreSQL Fine Tuning starting point
Qlik Sense Shared Storage – Options and Requirements
Qlik Help – Performance and Scalability
Q&A:
Q: Recently I'm facing Qlik Sense proxy servers RAM overload, although there are 4 nodes and each node it is 16 CPUs and 256G. We have done App optimazation, like delete duplicate app, remove old data, remove unused field...but RAM status still not good, what is next to fix the performace issue? Apply more nodes?
A: Depends on what you mean by “RAM status still not good”. Qlik Data Analytics software will allocate and use memory within the limits established and does not release this memory unless the Low Memory Limit has been reached and cache needs cleaning. If RAM consumption remains high but no other effects, your system is working as expected.
Q: Similar to other database, do you think we need to perform finetuning, cleaning up bad records within PostgresQL , e.g.: once per year?
A: Periodic cleanup, especially in a rapidly changing environment, is certainly recommended. A good starting point: set your Deleted Entity Log table cleanup settings to appropriate values, and avoid clean-up tasks kicking in before user morning rampup.
Q: Does QliKView Server perform similarly to Qlik Sense?
A: It uses the same QIX Engine for data processing. There may be performance differences to the extent that QVW Documents and QVF Apps are completely different concepts.
Q: Is there a simple way (better than restarting QS services)to clean the cache, because chache around 90 % slows down QS?
A: It’s not quite as simple. Qlik Data Analytics software (and by extent, your users) benefits from keeping data cached as long as possible. This way, users consume pre-calculated results from memory instead of computing the same results over and over. Active cache clearing is detrimental to performance. High RAM usage is entirely normal, based Memory Limits defined in QMC. You should not expect Qlik Sense (or QlikView) to manage memory like regular software. If work stops, this does not mean memory consumption will go down, we expect to receive and serve more requests so we keep as much cached as possible. Long winded, but I hope this sets better expectations when considering “bad performance” without the full technical context.
Q: How do we know when CPU hits 100% what the culprit is, for example too many concurrent user loading apps/datasets or mutliple apps qvds reloading? can we see that anywhere?
A: We will provide links to the Log Analysis app I demoed during the webinar, this is a great place to start. Set Repository Performance logs to DEBUG for the QRS performance part, start analysing service resource usage trends and get to know your user patterns.
Q: Can there be repository connectivity issues with too many nodes?
A: You can only grow an environment so far before hitting physical limits to communication. As a best practice, with every new node added, a review of QRS Connection Pools and DB connectivity should be reviewed and increased where necessary. The most usual problem here is: you have added more nodes than connections are allowed to DB or Repository Services. This will almost guarantee communication issues.
Q: Does qlik scalability tools measure browser rendering time as well or just works on API layer?
A: Excellent question, it only evaluates at the API call/response level. For results that include browser-side rendering, other tools are required (LoadRunner, complex to set up, expert help needed).
Transcript:
Hello everyone and welcome to the November edition of Techspert Talks. I’m Troy Raney and I’ll be your host for today's session. Today's presentation is Optimizing Performance for Qlik Sense Enterprise with Mario Petre. Mario why don't you tell us a little bit about yourself?
Hi everyone; good to be here with everybody once again. My name is Mario Petre. I’m a Principal Technical Engineer in the Signature Support Team. I’ve been with Qlik over six years now and since the beginning, I’ve focused on Qlik Sense Enterprise backend services, architecture and performance from the very inception of the product. So, there's a lot of historical knowledge that I want to share with you and hopefully it's an interesting springboard to talk about performance.
Great! Today we're going to be talking about how a Qlik Sense site looks from an architectural perspective; what are things that should be measured when talking about performance; what to monitor after going live; how to troubleshoot and we'll certainly highlight plenty of resources and where to find more details at the end of the session. So Mario, we're talking about performance for Qlik Sense Enterprise on Windows; but ultimately, it's software on a machine.
That's right.
So, first we need to understand what Qlik Sense services are and what type of resources they use. Can you show us an overview from what a multi-node deployment looks like?
Sure. We can take a look at how a large Enterprise environment should be set up.
And I see all the services have been split out onto different nodes. Would you run through the acronyms quickly for us?
Yep. On a consumer node this is where your users come into the Hub. They will come in via the Qlik Proxy Service and consume applications via the Qlik Engine Service, that ultimately connects to the central node and everything else via the Qlik Repository Service.
Okay.
The green box is your front-end services. This is what end users tap into to consume data, but what facilitates that in the background is always the Repository Service.
And what's the difference between the consumer nodes on the top and the bottom?
These two nodes have a Proxy Service that balances against their own engines as well as other engines; while the consumer nodes at the bottom are only there for crunching data.
Okay.
And then we can take a look at the backend side of things. Resources are used to the extent that you're doing reloads, you will have an engine there as well as the primary role for the central node, active and failover which is: the Repository Service to coordinate communication between all the rest of the services. You can also have a separate node for development work. And ultimately we also expect the size of an environment to have a dedicated storage solution and a dedicated central Repository Database host either locally managed or in one of the cloud providers like AWS RDS for example.
Between the front-end and back-end services where's the majority of resource consumption, and what resources do they consume?
Most of the resource allocation here is going to go to the Engine Service; and that will consume CPU and RAM to the extent that it's allocated to the machine. And that is done at the QMC level where you set your Working Set Limits. But in the case of the top nodes, the Proxy Service also has a compute cost as it is managing session connectivity between the end user's browser and the Engine Service on that particular server. And the Repository Service is constantly checking the authorization and permissions. So, ultimately front-end servers make use of both front-end and back-end resources. But you also need to think about connectivity. There is the data streaming from storage to the node where it will be consumed and then loading from that into memory. And these are three different groups of resources: you have compute; you have memory, and you have network connectivity. And all three have to be well suited for the task for this environment to work well.
And we're talking about speed and performance like, how fast is a fast network? How can we even measure that?
So, we would start for any Enterprise environment, we would start at a 10 Gb network speed and ultimately, we expect response time of 4 MS between any node and the storage back end.
Okay. So, what are some common bottlenecks and issues that might arise?
All right. So, let's take a look at some at some examples. The Repository Service failing to communicate with rim nodes, with local services. I would immediately try to verify that the Repository Service connection pool and network connectivity is stable and connect. Let's say apps load very very slow for the first time. This is where network speed really comes into play. Another example: the QMC or the Hub takes a very very long time to load. And for that, we would have to look into the communication between the Repository Service and the Database, because that's where we store all of this metadata that we will try to calculate your permissions based on.
And could that also be related to the rules that people have set up and the number of users accessing?
Absolutely. You can hurt user experience by writing complex rules.
What about lag in the app itself?
This is now being consumed by the Engine Service on the consumer node. So, I would immediately try to evaluate resource consumption on that node, primarily CPU. Another great example for is high Page File usage. We prefer memory for working with applications. So, as soon as we try to cache and pull those results again from disk, performance we'll be suffering. And ultimately, the direct connectivity. How good and stable is the network between the end users machine and the Qlik Sense infrastructure? The symptom will be on the end user side, but the root cause is almost always (I mean 99.9% of the time) will be down to some effect in the environment.
So, to get an understanding of how well the machine works and establish that baseline, what can we use?
One simple way to measure this (CPU, RAM, disk network) is this neat little tool called iPerf.
Okay. And what are we looking at here?
This is my central node.
Okay. And iPerf will measure what exactly?
How fast data transfer is between this central node and a client machine or another server.
And where can people find iPerf?
Great question. iPerf.fr
And it's a free utility, right?
Absolutely.
So, I see you've already got it downloaded there.
Right. You will have to download this package, both on the server and the client machine that you want to test between. We'll run this “As Admin.” We call out the command; we specify that we want it to start in “server mode.” This will be listening for connection attempts.
Okay.
We can define the port. I will use the default one. Those ports can be found in Qlik Help.
Okay.
The format for the output in megabyte; and the interval for refresh 5 seconds is perfectly fine. And then, we want as much output as possible.
Okay.
First, we need to run this. There we go. It started listening. Now, I’m going to switch to my client machine.
So, iPerf is now listening on the server machine and you're moving over to the client machine to run iPerf from there?
Right. Now, we've opened a PowerShell window into iPerf on the client machine. Then we call the iPerf command. This time, we're going to tell it to launch in “Client Mode.” We need to specify an IP address for it to connect to.
And that's the IP address of the server machine?
Right. Again, the port; the format so that every output is exactly the same. And here, we want to update every second.
Okay.
And this is a super cool option: if we use the bytes flag, we can specify the size of the data payload. I’m going to go with a 1 Gb file (1024 Mb). You can also define parallel connections. I want 5 for now.
So, that's like 5 different users or parallel streams of activity of 1 Gb each between the server machine and this client machine?
Right. So, we actually want to measure how fast can we acquire data from the Qlik Sense server onto this client machine. We need to reverse the test. So, we can just run this now and see how fast it performs.
Okay. And did the server machine react the same way?
You can see that it produced output on the listening screen. This is where we started. And then it received and it's displaying its own statistics. And if you want to automate this, so that you have a spot check of throughput capacity between these servers, we need to use the log file option. And then we give it a path. So, I’m gonna say call this “iperf_serverside…” And launch it. And now, no output is produced.
Okay.
So, we can switch back to the client machine.
Okay. So, you're performing the exact same test again, just storing everything in a log file.
The test finished.
Okay. So, that can help you compare between what's being sent to what's being received, and see?
Absolutely. You can definitely have results presented in a way that is easy to compare across machines and across time. And initial results gave us a throughput per file of around 43.6, 46, thereabouts megabytes per second.
So, what about for an end user who's experiencing issues? Can you use iPerf to test the connectivity from a user machine on a different network?
Yep. So, in in the background we will have our server; it's running and waiting for connections. And let's run this connection now from the client machine. We will make sure that the IP address is correct; default port; the output format in megabytes; we want it refreshed every second; and we are transferring 1 Gb; and 5 parallel streams in reverse order. Meaning: we are copying from the server to the client machine. And let's run it.
Just seeing those numbers, they seem to be smaller than what we're seeing from the other machine.
Right. Indeed. I have some stuff in between to force it to talk a little slower. But this is one quick way to identify a spotty connection. This is where a baseline becomes gold; being able to demonstrate that your platform is experiencing a problem. And to quantify and to specify what that problem is going to reduce the time that you spend on outages and make you more effective as an admin.
Okay. That was network. How can admins monitor all the other performance aspects of a deployment? What tools are available and what metrics should they be measuring?
Right. That's a great question. The very basic is just Performance Monitor from Windows.
Okay.
The great thing about that is that we provide templates that also include metrics from our services.
Can you walk us through how to set up the Performance Monitor using one of those templates?
Sure thing. So, we're going to switch over first to the central node. So, the first thing that I want to do is create a folder where all of these logs will be stored.
Okay. So, that's a shared folder, good.
And this article is a great place to start. So, we can just download this attachment
So, now it's time to set up a Performance Monitor proper. We need to set up a new Data Collector Set.
Giving it a name.
And create from template. Browse for it, and finish.
Okay. So it’s got the template. That's our new one Qlik Sense Node Monitor, right?
Yep. You'll have multiple servers all writing to the same location. The first thing is to define the name of each individual collector; and you do that here. And you can also provide subdirectory for these connectors, and I suggest to have one per node name. I will call this Central Node.
Everything that comes from this node, yeah.
Correct. You can also select a schedule for when to start these. We have an article on how to make sure that Data Collectors are started when Windows starts. And then a stop condition.
Now, setting up monitors like this; could this actually impact performance negatively?
There is always an overhead to collecting and saving these metrics to a file. But the overhead is negligible.
Okay.
I am happy with how this is defined. Now, this static collector on one of the nodes is already set up. There is an option here that's called Data Manager. What's important here to define is to set a Minimum Free Disk. We could go with 10 Gb, for example; and you can also define a Resource Policy. The important bit is Minimum Free Disk. We want to Delete the Oldest (not the largest) in the Data Collector itself. We should change that directory and make sure that it points to our central location instead of locally; and we'll have to do this for every single node where we set this up.
Okay. So, that's that shared location?
Yep.
And you run the Data Collector there. And it creates a CSV file with all those performance counters. Cool.
So, here we have it now. If we just take a very quick look inside, we'll see a whole bunch of metrics. And if you want to visualize these really really quick, I can show you a quick tip that wasn't on the agenda but since we're here: on Windows, there is a built-in tool called Relog that is specifically designed for reformatting Performance Monitor counters. So, we can use Relog; we'll give it the name of this file; the format will be Binary; the output will be the same, but we'll rename it to BLG; and let's run it.
And now it created a copy in Binary format. Cool thing about this Troy is that: you can just double click on it.
It's already formatted to be a little more readable. Wow! Check that out.
There we go. Another quick tip: since we're here, first thing to do is: select everything and Scale; just to make sure that you're not missing any of the metrics. And this is also a great way to illustrate which service counters and system counters we collect. As you can see, there's quite a few here.
Okay. So, that Performance Monitor is, it's set up; it's running; we can see how it looks; and that is going to run all the time or just when we manually trigger it?
You can definitely configure it to run all the time, and that would be my advice. Its value is really realized as a baseline.
Yeah. Exactly. That was pretty cool seeing how that worked, using all the built-in utilities. And that Relog formatting for the Process Monitor was new to me. Are there any other tools you like to highlight?
Yeah. So, Performance Monitor is built-in. For larger Enterprises that may already be monitoring resources in a centralized way, there's no reason why you shouldn't expect to include the Sense resources into that live monitoring. And this could be done via different solutions out there. A few come to mind like: Grafana, Datadog, Butler SOS, for example from one of our own Qlik luminaries.
Can we take a quick look at Grafana? I’ve heard of that but never seen it.
Sure thing. This is my host monitor sheet. It's nowhere built to a corporate standard, but you can see here I’m looking at resources for the physical host where these VMs are running as well as the domain controller, and the main server where we've been running our CPU tests. And the great part about this is I have historical data as far back I believe as 90 days.
So, this is a cool tool that lets you like take a look at the performance and zoom-in and find the processes that might be causing some peaks or anything you want to investigate?
Right. Exactly. At least come up with a with a narrow time frame for you to look into the other tools and again narrow down the window of your investigation.
Yeah, that could be really helpful. Now I wanted to move on to the Qlik Sense Scalability Tools. Are those available on Qlik community?
That's right. Let me show you where to find them. You can see that we support all current versions including some of the older ones. You will have to go through and download the package and the applications used for analysis afterwards. There is a link over here. So, once the package is downloaded, you will get an installer. And the other cool thing about Scalability Tools is that you can use it to pre-warm the cache on certain applications since Qlik Sense Enterprise doesn't support application pre-loading.
Oh, cool. So, you can throttle up applications into memory like in QlikView. Can we take a look at it?
Yes, absolutely. This is the first thing that you'll see. We'll have to create a new connection. So, I’ll open a simple one that I’ve defined here and we can take a look at what's required just to establish a quick connection to your Qlik Sense site.
Okay, but basically the scenario that you're setting up will simulate activity on a Qlik Sense site to test its performance?
Exactly. You'll need to define your server hostname. This can be any of your proxy nodes in the environment. The virtual proxy prefix. I’ve defined it as Header and authentication method is going to be WebSocket.
Okay.
And then, if we want to look at how virtual users are going to be injected into the system, scroll over here to the user section. Just for this simple test, I’ve set it up for User List where you can define a static list of users like so: User Directory and UserName.
Okay. So, it's going to be taking a look at those 2 users you already predefined and their activity?
Exactly. We need to test the connection to make sure that we can connect to the system. Connection Successful. And then we can proceed with the scenario. This is very simple but let me show you how I got this far. So, the very first thing that we should do is to Open an App.
So, you're dragging away items?
Yep. I’m removing actions from this list. Let's try to change the sheet. A very simple action. And now we have four sheets, and we'll go ahead and select one of them.
Okay, so far, we have Opening the App and immediately changing to a sheet?
Yep. That's right. This will trigger actions in sequence exactly how you define them. It will not take into consideration things like Think Time. I will just define a static weight of 15 seconds, and then you can make selections.
But this is an amazing tool for being able to kind of stress test your system.
It's very very useful and it also provides a huge amount of detail within the results that it produces. One other quick tip: while defining your scenario, use easy to read labels, so that you can identify these in the Results Application. Let's assume that the scenario is defined. We will go ahead and add one last action and that is: to close, to Disconnect the app. We'll call this “OpenApp.” We'll call this “SheetChange.” Make sure you Save. The connection we've tested; we've defined our list of users. First, let's run the scenario. There is one more step to define and that is: to configure an Executor that will use this scenario file to launch a workload against our system. Create a New Sequence.
This is just where all these settings you're defining here are saved?
Correct. This is simply a mapping between the execution job that you're defining and which script scenario should be used. We'll go ahead and grab that. Save it again; and now we can start it. And now in the background if we were to monitor the Qlik Sense environment, we would see some amount of load coming in. We see that we had some kind of issue here: empty ObjectID. Apparently I left something in the script editor; but yeah, you kind of get the idea.
So, all this performance information would then be loaded into an app that is part of the package downloaded from Qlik community. How does that look?
So, here you will see each individual result set, and you can look at multiple-exerciser runs in the single application. Unfortunately, we don't have more than one here to showcase that, but you would see multiple-colored lines. There is metrics for a little bit of everything: your session ramp, your throughput by minute, you can change these.
CPU, RAM. This is great.
Exactly. CPU and RAM. These are these are not connected. We don't have those logs, but you would have them for a setup run on your system. These come from Performance Monitor as well, so you could just use those logs provided that the right template is in place. We see Response Time Distribution by Action, and these are the ones that I’ve asked you to change and name so that they're easy to understand.
Once your deployment is large enough to need to be multi-node and the default settings are no longer the best ones for you, what needs to be adjusted with a Repository Service to keep it from choking or to improve its performance?
That's a great question Troy. So, the first thing that we should take a look at is how the Repository communicates with the backend Database and vice versa. The connection pool for the Repository is always based on core count on the machine. And the best rule of thumb that we have to date is to take your core count on that machine, multiply it by 5, and that will be the max connection pool for the Repository Service for that node.
Can you show us where that connection pool setting can be changed?
Yes. So, we will go ahead and take a look. Here we are on the central node of my environment. You'll have to find your Qlik installation folder. We'll navigate to the Repository folder, Util, QlikSenseUtil, and we'll have to launch this “As Admin.”
Okay.
We'll have to come to the Connection String Editor. Make sure that the path matches. We just have to click on Read so that we get the contents of these files. And the setting that we are about to change is this one.
Okay. So, the maximum number of connections that the Repository can make?
Yes. And this is (again) for each node going towards the Repository Database.
Okay.
Again, this should be a factor of CPU cores multiplied by 5. If 90 is higher than that result, leave 90 in place. Never decrease it.
Okay, that's a good tip.
Right. I change this to 120. I have to Save. What I like to do here is: clear the screen and hit Read again; just to make sure that the changes have been persisted in the file.
Okay.
Once that's done, we can close this. We can restart the environment. We can get out of here.
So, there you adjusted the setting of how many connections this node can make to the QSR. Then assuming we do the same on all nodes, where do we adjust the total number of connections the Repository itself can receive?
That should be a sum of all of the connection strings from all of your nodes plus 110 extra for the central node. By default, here is where you can find that config file: Repository, PostgreSQL, and we'll have to open this one, PostgreSQL. Towards the end of the file…
Just going all the way to the bottom.
Here we have my Max Connections is 300.
Okay. One other setting you mentioned was the Page File and something to be considered. How would we make changes or adjust that setting?
Right. So, this is a Windows level setting that's found in Advanced System Settings; Advanced tab; Performance; and then again Advanced; and here we have Virtual Memory.
Okay.
We have to hit Change. We'll have to leave it at System Managed or understand exactly which values we are choosing and why. If you're not sure, the default should always be System Managed.
Now, I want to know what resources are available for Qlik Sense admins; specifically, what is the Admin Playbook?
It's a great starting place for understanding what duties and responsibilities one should be thinking about when administering a Qlik Sense site.
So, these are a bunch of tools built by Qlik to help analyze your deployment in different ways. I see weekly, monthly, quarterly, yearly, and a lot of different things are available there.
Yeah. So, we can take a look at Task Analysis, for example. The first time you run it, it's going to take about 20 minutes; thereafter about 10. The benefits: it shows you really in depth how to get to the data and then how to tweak the system to work better based on what you have.
Yeah, that's great.
Right? So, not only we put the tools in your hands, but also how to build these tools as you can here. See here, we have instructions on how to come up with these objects from scratch. An absolute must-read for every system admin out there.
Mario, we've talked about optimizing the Qlik Sense Repository Service, but not about Postgres? Do larger Enterprise level deployments affect its performance?
Sure. The thing about Postgres is again: we have to configure it by default for compatibility and not performance. So, it's another component that has to be targeted for optimization.
The detail there that anything over 1 Gb from Postgres might get paged - that sounds like it could certainly impact performance.
Right, because the buffer setting that we have by default is set to 1 Gb; and that means only 1 Gb of physical memory will be allocated to Postgres work. Now, we're talking about the large environment 500 to maybe 5,000 apps. We're talking 1000s of users with about 1000 of them peak concurrency per hour.
So, can we increase that Shared Buffer setting?
Absolutely. And in fact, I want to direct you to a really good article on performance optimization for PostgreSQL. And when we talk about fine-tuning, this article is where I’d like to get started. We talk about certain important factors like the Shared Buffers. So, this is what we define to 1 Gb by default. Their recommendation is to start with 1/4 of physical memory in your system. 1 Gb is definitely not one quarter of the machines out there. So, it needs tweaking.
And again these are settings to be changed on the machine that's hosting the Repository Database, right?
That's correct. That's correct.
Now, is there an app that you're aware of that would be good to kind of look at all these logs and analyze what's going on with the performance?
Absolutely. This is an application that was developed to better understand all of the transactions happening in a particular environment. It reads the log files collected with the Log Collector either via the tool or the QMC itself.
Okay.
It's not built for active monitoring, but rather to enhance troubleshooting.
Sure. So, basically it's good for looking at a short period of time to help troubleshooting?
Right. The Repository itself communicates over APIs between all the nodes and keeps track of all of the activities in the system; and these translate to API calls. If we want to focus on Repository API calls, we can start by looking at transactions.
Okay.
So, this will give us detail about cost. For example, per REST call or API call, we can see which endpoints take the most, duration per user, and this gives you an opportunity to start at a very high level and slowly drill in both in message types and timeframe. Another sheet is the Threads Endpoints and Users; and here you have performance information about how many worker-threads the Repository Service is able to start, what is the Repository CPU consumption, so you can easily identify one. For example, here just by discount, we can see that the preview privileges call for objects is called…
Yeah, a lot.
Over half a million times, right? And represents 73% of the CPU compute cost.
Wow, nice insights.
And then if we look here at the bottom, we can start evaluating time-based patterns and select specific time frames and go into greater detail.
So, I’m assuming this can also show resource consumption as well?
Right. CPU, memory in gigabytes and memory in percent. One neat trick is: to go to the QMC, look at how you've defined your Working Set Limits, and then pre-define reference lines in this chart. So, that it's easier to visualize when those thresholds are close to being reached or breached. And you do that by the add-ons reference lines, and you can define them like this.
That's just to sort of set that to match what's in the QMC?
Exactly.
Makes a powerful visualization. So, you can really map it.
Absolutely. And you can always drill down into specific points in time we can go and check the log details Engine Focus sheet; and this will allow us to browse over time, select things like errors and warnings alone, and then we will have all of the messages that are coming from the log files and what their sources.
Yeah. That's great to have it all kind of collected here in one app, that's great.
Indeed.
To summarize things, we've talked about to understand system performance, a baseline needs to be established. That involves setting up some monitoring. There are lots of options and tools available to do that; and it's really about understanding how the system performs so the measurement and comparisons are possible if things don't perform as expected.
And to begin to optimize as well.
Okay, great. Well now, it's time for Q&A. Please submit your questions through the Q&A panel on the left side of your On24 console. Mario, which question would you like to address first?
We have some great questions already. So, let's see - first one is: how can we evaluate our existing Qlik Sense applications?
This is not something that I’ve covered today, but it's a great question. We have an application on community called App Metadata Analyzer. You can import this into your system and use it to understand the memory footprint of applications and objects within those applications and how they scale inside your system. It will very quickly illustrate if you are shipping applications with extremely large data files (for example) that are almost never used. You can use that as a baseline for both optimizing local applications and also in your efforts to migrating to SaaS, if you feel like you don't want to bother with all of this Performance Monitoring and optimization, you can always choose to use our services and we'll take care of that for you.
Okay, next question.
So, the next question: worker schedulers errors and engine performance. How to fix?
I think I would definitely point you back to this Log Analysis application. Load that time frame where you think something bad happened, and see what kind of insights you can you can get by playing with the data, by exploring the data. And then narrow that search down if you find a specific pattern that seems like the product is misbehaving. Talk to Qlik support. We'll evaluate that with you and determine whether this is a defect or not or if it's just a quirk of how your system is set up. But that Sense Log Analysis app is a great place to start. And going back to the sheet that I showed: Repository and Engine metrics are all collected there. And these come from the performance logs that we already produce from Qlik Sense. You don't need to load any additional performance counters to get those details.
Okay.
All right. So, there is a question here about Postgres 9.6 and the fact that it's soon coming end of life. And I think this is a great moment to talk about this. Qlik Sense client-managed or Qlik Sense Enterprise for Windows supports Postgres 12.5 for new installations since the May release. If you have an existing installation, 9.6 will continue to be used; but there is an article on community on how to in-place upgrade that to 12.5 as a standalone component. So, you don't have to continue using 9.5 if your IT policy is complaining about the fact that it's soon coming to the end of life. As we say, we are aware of this fact; and in fact, we are shipping a new version as of the May 2021 release.
Oh, great.
So, here's an interesting question. If we have Qlik Sense in Azure on a virtual machine, why is the performance so sluggish? How do you fine-tune it? I guess first we need to understand what would you mean by sluggish? But the first thing that I want to point to is: different instance types. So, virtual machines in virtual private cloud providers are optimized for different workloads. And the same is true for AWS, Azure and Google Cloud platform. You will have virtual machines that are optimized for storage; ones that are optimized for compute tasks or application analytics; some that are optimized for memory. Make sure that you've chosen the right instance type and the right level of provisioned iOps for this application. If you feel that your performance is sluggish, start increasing those resources. Go one tier up and reevaluate until you find a an instance type that works for you. If you wish to have these results (let's say beforehand), you will have to consider using the Scalability Tools together with some of your applications against different instance types in Azure to determine which ones work best.
Just to kind of follow up on that question, if we're looking at that multi-node example from Qlik help, what nodes would you consider would require more resources?
Worker nodes in general. And those would be front and back-end.
So, a worker node is something with an engine, right?
Exactly. Something with an engine. It can either be front-facing together with a proxy to serve content, or back-end together with a scheduler a service to perform reload tasks. These will consume all the resources available on a given machine.
Okay.
And this is how the Qlik Sense engine is developed to work. And these resources are almost never released unless there is a reason for it, because us keeping those results cached is what makes the product fast.
Okay.
Oh, here's a great one about avoiding working set breaches on engine nodes. Question says: do you have any tips for avoiding the max memory threshold from the QIX engine? We didn't really cover this this aspect, but as you know the engine allows you to configure memory limits both for the lower and higher memory limit. Understanding how these work; I want to point you back to that QIXs engine white paper. The system will perform certain actions when these thresholds are reached. The first prompt that I have for you in this situation is: understand if these limits are far away from your physical memory limit. By default, Qlik Sense (I believe) uses 70 / 90 as the low and high working sets on a machine. With a lot of RAM, let's say 256 - half a terabyte of RAM, if you leave that low working set limit to 70 percent, that means that by default 30 of your physical RAM will not be used by Qlik Sense. So. always keep in mind that these percentages are based on physical amount of RAM available on the machine, and as soon as you deploy large machines (large: I’m talking 128 Gb and up) you have to redefine these parameters. Raise them up so that you utilize almost all of the resources available on the machine ,and you should be able to visualize that very very easily in the Log Analysis App by going to Engine Load sheet and inserting those reference lines based on where your current working sets are. Of course, the only way really to avoid a working set limit issue is to make sure that you have enough resources. And the system is configured to utilize those resources, so even if you still get them after raising the limit and allowing the - allowing the product to use as much RAM as it can without of course interfering with Windows operations (which is why you should never set these to like 99, 98, 99). Windows needs RAM to operate by itself, and if we let Qlik Sense to take all of it, it will break things. If you've done that and you're still having performance issues, that means you need more resources.
Yeah. It makes sense.
Oh, so here is another interesting question about understanding what certain Qlik Repository Service (QRS) log messages say. There is a question here that says: try to meet the recommendation of network and persistence the network latency should be less than 4 MS, but consistently in our logs we are seeing the QRS security management retrieved privileges in so many milliseconds. Could this be a Repository Service issue or where would you suggest we investigate first? This is an info level message that you are reporting. And it's simply telling you how long it took for the Repository Service to compute the result for that request. That doesn't mean that this is how long it took to talk to the Database and back, or how long it took for the request to reach from client to the server; only how long it took for the Repository Service to look up the metadata look up the security rules and then return a result based on that. And I would say this coming back in 384 milliseconds is rather quick. It depends on how you've defined these security rules. If these security rules are super simple and you are still getting slow responses, we would definitely have to look at resource consumption. But if you want to know how these calls affect resource consumption on the Repository and Postgres side, go back to that Log Analysis App. Raise your Repository performance logs in the QMC to Debug levels so that you get all of the performance information of how long each call took to execute. And try to establish some patterns. See if you have calls that take longer to execute than others; and where are those coming from any specific apps, any specific users? All of these answers come from drilling down into the data via that app that I demoed.
Okay Mario, we have time for one last question.
Right. And I think this is an excellent one to end. We talked a whole bunch here about Qlik Sense, but all of this also applies to QlikView environments. We are always looking at taking a step back and considering all of the resources that are playing in the ecosystem, not just the product itself. And the question asks: is QlikView Server performance similar to how it handles resources Qlik Sense? The answer is: yes. The engine is exactly the same in both products. If you read that white paper, you will understand how it works in both QlikView and Qlik Sense. And the things that you should do to prepare for performance and optimization are exactly the same in both products. Excellent question.
Great. Well, thank you very much Mario!
Oh, it's been my pleasure Troy. That was it for me today. Thank you all for participating. Thank you all for showing up. Thank you Troy for helping me through this very very complicated topic. It's been a blast as always. And to our customers and partners, looking forward to seeing your questions and deeper dives into logs and performance on community.
Okay, great! Thank you everyone! We hope you enjoyed this session. Thank you to Mario for presenting. We appreciate getting experts like Mario to share with us. Here's our legal disclaimer and thank you once again. Have a great rest of your day.
From Qlik Replicate 2023.5 onwards, testing the HTTPS connection may fail with:
HTTPS Test of HTTPS configuration failed - the server cannot accept secure connections. Fix the problem or set testHttps to false and restart the server
Qlik Replicate 2023.5 and above
Qlik Replicate 2023.5 and above no longer have a serviceconfiguration.xml to edit.
In Qlik Sense, there could be a situation where the server is installed and functioning, all the services are running except the fact that all the scheduler tasks are failing.
The scheduler service may fail after restarting. This indicates that the issue is related to the scheduler ExecutionSessions table data in the Repository PostGres SQL database.
In the Qlik Sense Management Console, task statuses may be listed as Reset.
This issue happens when users add, remove, delete too many tasks while frequently stopping and starting the scheduler service.
The data in the scheduler tasks execution sessions table gets corrupted. The startup sequence will start to fail and cause errors in the scheduler logs.
Qlik Sense Enterprise on Windows
Corruption may occur due to:
This can look like this in the repository database:
The following will be printed to the scheduler logs:
System.Scheduler.Scheduler.Core.Utility.SystemInformationProvider
An item with the same key has already been added.
and/or
System.Scheduler.Scheduler.Main
Startup sequence failed, shutting down to avoid any inconsistent state, please restart the scheduler
and/or
Unexpected exception in Delete for Entity ExecutionSession with Id <Task ID>
To resolve this issue, it is recommended to wipe the data from the Execution sessions table.
Please note that this will need to be a hard deletion from the table and not soft deletion.
SELECT
"ExecutionSessions"."ID",
"ExecutionSessions"."AbortAt",
"ExecutionSessions"."Retries",
"ExecutionSessions"."AbortReason",
"ExecutionSessions"."User",
"ExecutionSessions"."Deleted",
"ExecutionSessions"."CreatedDate",
"ExecutionSessions"."ModifiedDate",
"ExecutionSessions"."SyncDate",
"ExecutionSessions"."ModifiedByUserName",
"ExecutionSessions"."App_ID",
"ExecutionSessions"."ExecutingNode_ID",
"ExecutionSessions"."ExecutionResult_ID",
"ExecutionSessions"."ReloadTask_ID",
"ExecutionSessions"."UserSyncTask_ID",
"ExecutionSessions"."ExternalProgramTask_ID"
FROM
public."ExecutionSessions";
DELETE
FROM
public."ExecutionSessions";
After installing Qlik Sense Desktop February 2024 or upgrading from a previous version, the display scaling no longer works as expected.
Blank space is shown to the right and bottom of the Qlik Sense client:
This issue is caused by QB-25016. A fix will be deployed with the Qlik Sense Desktop May 2024 release.
Qlik Sense Desktop May 2024
Product Defect ID: QB-25016
Security of Qlik Sense Enterprise on Windows can be approached in the below discrete areas. All these areas provide different options for increasing security in a deployment, and thereby mitigating vulnerabilities and protecting against attackers.
Content:
Be aware that a high level of server hardening can lead to failure in your deployment. Be mindful of always having a backup to restore to in case your configuration leads to irreversible failure.
Qlik Sense Enterprise on Windows supports multiple different Authentication Solutions;
Qlik can not specify which authentication method is appropriate for each deployment. It is advisable to review currently supported alternatives within your organization and/or Identity Provider (IdP) to implement the most suitable solution for your use case.
Qlik Sense Enterprise on Windows provides two levels of native authorization in the product.
Attribute based access control (ABAC), which is configured through Qlik Sense security rules. This article will not go in depth on how to best implement security rules for your requirements, but it is highly recommended to think of your users based on the capabilities that you intend to provide them. For example different roles and capabilities as shown in image below, allows for a security rule framework to be designed and implemented. This can be done either by yourself by referencing Qlik Sense Help for Administrators and available assets or by engaging with a Qlik Consultant or Qlik Partner.
Row level data reduction, which is configured through Section Access at Qlik Sense app level. This article will not go in depth on Section Access implementation, but with this reduction a single file can be used to hold the data for a number of users or user groups. Qlik Sense uses the information in the section access for authentication and authorization, and dynamically reduces the data, so that users only see their own data.
Qlik Sense Enterprise on Windows inherits the available protocols, cipher suites, key exchanges and other security hardening which are enabled on the Windows Server operating Qlik Sense.
Windows Server has a lot of protocols enabled by default; however protocols, ciphers, hashes and key exchanges that are considered deprecated or not secure enough should be disabled. There are many ways of doing this, and the Windows administrator and security experts should be consulted so that local policies are accurately applied. For simplicity, understanding and a good overview IIS Crypto 3.0 can be a good tool for evaluating current Windows configuration and applying changes.
Keep in mind that "Best Practice" today might not be recommended in the near future, what was considered "safe" a while ago is not necessarily considered so today. For this reason, it is also important to regularly scan servers for potential vulnerabilities and revisit configurations as required.
Windows Server needs to be restarted for these settings changes to take effect. It is also important to ensure that all components running on the server still operate as expected after hardening is applied, for example older non-Qlik software might not be compliant with the latest options and standards.
Firewalls typically should be closed, with required ports only opened for intended purposes.
See Qlik Sense Enterprise on Windows ports overview for details on required port based on the deployed architecture.
For most organizations, local administrator rights allow for an easier deployment, but Qlik Sense Enterprise on Windows does not require local administrator rights in order to function. This can be an attractive option inside some organizations. This will require additional configuration of boot strap mode as described in Qlik Sense Enterprise on Windows Services.
For a brief overview of the rights needed by a Qlik Sense Enterprise service account:
Qlik Sense Enterprise for Windows does not officially support Group Managed Service Accounts (gMSA), but it can operate using one. The initial barrier is that the installer requires a service account and password to be entered during installation. A domain or local account could be substituted for the install stages only to be swapped out in the Windows Services applet (services.msc) after installation. Some functionality may require workarounds (e.g. A User Directory Connection to Active Directory).
Qlik Sense Enterprise on Windows does require exceptions from anti-virus scan to avoid potential disk I/O conflicts. Refer to Qlik Sense Folder And Files To Exclude From AntiVirus Scanning for more details.
Qlik Sense Enterprise on Windows can run with Federal Information Processing Standards (FIPS) enabled on the Windows Server. This does require a few adjustments of configuration files due to Qlik using non-FIPS compliant algorithms for minor tasks like hash checks. See Running Qlik Sense on Windows systems with FIPS compliance enabled for more details on Qlik Sense and FIPS.
Qlik Sense Enterprise on Windows uses PostgreSQL to store meta-data relating to a Qlik Sense site. In multi-node sites or sites where PostgreSQL is isolated from Qlik Sense Enterprise for Windows additional security can be applied;
Qlik Sense Proxy service bundled with Qlik Sense Enterprise on Windows is simply a web-service. This means applying general practice guidance but in the context of Qlik Sense as described below.
Qlik Sense Enterprise on Windows acts as a Certificate Authority (CA) during initial installation and signs a certificate that is applied on all encrypted traffic between Qlik Sense services. The same Qlik Sense signed certificate is applied as default certificate also for incoming connections from users accessing Qlik Sense Hub and QMC. This default certificate is not intended for production use, unless user access to Qlik Sense comes through a network load balancer or reverse proxy that trusts the Qlik Sense certificate. For direct user access to Qlik Sense Proxy, a fully trusted certificate can typically be acquired from your local IT and then applied on the Qlik Sense Proxy service.
As of July 2019, Qlik Sense Enterprise on Windows support SHA1 and SHA2 certificates. If SHA384 or SHA512 certificates are needed, then a network load balancer or reverse proxy can be configured in front of Qlik Sense which offloads to Qlik Sense.
There are numerous HTTP response headers that can be used in attempting to secure a server. Below are a couple of the most common ones, but as always it is recommended to consult local IT and web security expert on what the recommendations are.
Any additional HTTP response header values can be configured in Qlik Sense Virtual Proxy settings under Additional response headers as shown in the below image and described in Qlik Sense for Administrators: Virtual Proxies. It is recommended to trial any header changes in a new virtual proxy, as poor configuration may accidentally lock you out from Qlik Sense access.
Policy is a placeholder for your policy of choice and cannot be used as a value. See Writing a Policy (Mozilla) for examples.
Some connectors require an encryption key before you create or edit a connection. Failing to generate a key will result in:
Error retrieving the URL to authenticate: ENCRYPTION_KEY_MISSING - you must manually set an encryption key before creating new connections.
Qlik Sense Desktop February 2022 and onwards
Qlik Sense Enterprise on Windows February 2022 and onwards
all Qlik Web Storage Provider Connectors
Google Drive and Spreadsheets Metadata
PowerShell demo on how to generate a key:
# Generates a 32 character base 64 encoded string based on a random 24 byte encryption key
function Get-Base64EncodedEncryptionKey {
$bytes = new-object 'System.Byte[]' (24)
(new-object System.Security.Cryptography.RNGCryptoServiceProvider).GetBytes($bytes)
[System.Convert]::ToBase64String($bytes)
}
$key = Get-Base64EncodedEncryptionKey
Write-Output "Get-Base64EncodedEncryptionKey: ""${key}"", Length: $($key.Length)"
Example output:
Get-Base64EncodedEncryptionKey: "muICTp4TwWZnQNCmM6CEj4gzASoA+7xB", Length: 32
This command must be run by the same user that is running the Qlik Sense Engine Service (Engine.exe). For Qlik Sense Desktop, this should be the currently logged-in user.
Do the following:
Open a command prompt and navigate to the directory containing the connector .exe file. For example:
"cd C:\Program Files\Common Files\Qlik\Custom Data\QvWebStorageProviderConnectorPackage"
Run the following command:
QvWebStorageProviderConnectorPackage.exe /key {key}
Where {key} is the key you generated. For example, if you used the OpenSSL command, your key might look like: QvWebStorageProviderConnectorPackage.exe /key zmn72XnySfDjqUMXa9ScHaeJcaKRZYF9w3P6yYRr
You will receive a confirmation message:
Info: Set key. New key id=qseow_prm_custom.
Info: key set successfully!
The {sense service user} must be the name of the Windows account which is running your Qlik Sense Engine Service. You can see this in the Windows Services manager. In this example, the user is: MYCOMPANY\senseserver.
Do the following:
Open a command prompt and run:
runas /user:{sense service user} cmd. For example:runas /user:MYCOMPANY\senseserver
Run the following two commands to switch to the directory containing the connectors and then set the key:
"cd C:\Program Files\Common Files\Qlik\Custom Data\QvWebStorageProviderConnectorPackage"
QvWebStorageProviderConnectorPackage.exe /key {key}
Where {key} is the key you generated. For example, if you used the OpenSSL command, your key might look like: QvWebStorageProviderConnectorPackage.exe /key zmn72XnySfDjqUMXa9ScHaeJcaKRZYF9w3P6yYRr
You should repeat this step, using the same key, on each node in the multinode environment.
Encryption keys will be stored in: "C:\Users\{sense service user}\AppData\Roaming\Qlik\QwcKeys\"
For example, encryption keys will be stored in "C:\Users\QvService\AppData\Roaming\Qlik\QwcKeys\"
Always run the command prompt while logged in with the Qlik Sense Service Account which is running your Qlik Sense Engine Service and which has access to all the required folders and files.
This security requirement came into effect in February 2022. Old connections made before then will still work, but you will not be able to edit them. If you try to create or edit a connection that needs a key, you will receive an error message: Error retrieving the URL to authenticate: ENCRYPTION_KEY_MISSING) - you must manually set an encryption key before creating new connections.
This article provides step-by-step instructions for implementing Azure AD as an identify provider for Qlik Cloud. We cover configuring an App registration in Azure AD and configuring group support using MS Graph permissions.
It guides the reader through adding the necessary application configuration in Azure AD and Qlik Sense Enterprise SaaS identity provider configuration so that Qlik Sense Enterprise SaaS users may log into a tenant using their Azure AD credentials.
Content:
Throughout this tutorial, some words will be used interchangeably.
The tenant hostname required in this context is the original hostname provided to the Qlik Enterprise SaaS tenant.
Copy the "value of the client secret" and paste it somewhere safe.After saving the configuration the value will become hidden and unavailable.
In the OpenID permissions section, check email, openid, and profile. In the Users section, check user.read.
Failing to grant consent to GroupMember.Read.All may result in errors authenticating to Qlik using Azure AD. Make sure to complete this step before moving on.
In this example, I had to change the email claim to upn to obtain the user's email address from Azure AD. Your results may vary.
While not hard, configuring Azure AD to work with Qlik Sense Enterprise SaaS is not trivial. Most of the legwork to make this authentication scheme work is on the Azure side. However, it's important to note that without making some small tweaks to the IdP configuration in Qlik Sense you may receive a failure or two during the validation process.
For many of you, adding Azure AD means you potentially have a bunch of clean up you need to do to remove legacy groups. Unfortunately, there is no way to do this in the UI but there is an API endpoint for deleting groups. See Deleting guid group values from Qlik Sense Enterprise SaaS for a guide on how to delete groups from a Qlik Sense Enterprise SaaS tenant.
Qlik Cloud: Configure Azure Active Directory as an IdP
This article describes cannot reach host error when registering or re-register a Rim node.
Cannot reach the following remote host when distributing certificates: http://nodename:4444/setup/certificateDistribution
Environment:
What are the possible causes?
1. Make sure Central and Rim nodes have been patched to the same level.
2. See related article Qlik Sense - Node is offline - Failed to retrieve service status from 'http://Servername:4444/status/' (Unable to connect to the remote server)
3. See related article Rim node offline in QMC, bootstrap shows "Waiting for initial configuration to be run by node"
4. To be able to re-register a deleted RIM node, it is necessary to uninstall the Qlik Sense on RIM node and reinstall it before the node re-registration.
The "Remove Qlik Sense certificates and data folders" option should be selected when uninstall the Qlik Sense Enterprise Server. See Deleting Nodes as the reference.
Note:
If the reinstall of the RIM node doesn't solve the issue, it could be an issue from the Central node being corrupted.
In that case the solution would be to repair it:
Control Panel --> Programs and Features --> click "Change" on Qlik Sense and then Repair
By default, Qlik Replicate control tables like attrep_apply_exceptions or attrep_status are created in lower case. If you would like for those tables to be created in UPPERCASE on Snowflake as a target, please see below.
Starting in version 2021.5 service pack 02, there is an internal parameter called setIgnoreCaseFlag which if checked or set to true, will set the QUOTED_IDENTIFIERS_IGNORE_CASE to true.
Thus, letters in double-quoted identifiers are stored and resolved as uppercase letters.
For reference: https://docs.snowflake.com/en/sql-reference/parameters.html#quoted-identifiers-ignore-case
You can search for setIgnoreCaseFlag in the Advanced tab -> Internal Parameters for the Snowflake Target endpoint.
The information in this article is provided as-is and to be used at own discretion. Depending on tool(s) used, customization(s), and/or other factors ongoing support on the solution below may not be provided by Qlik Support.
When logging out of Qlik Cloud using AWS Cognito as an OIDC Identity Provider, the following error is displayed:
Required String parameter 'client_id' is not present
AWS Cognito requires client_id and logout_uri as string parameters but Qlik Cloud does not send those as parameters at logout time.
For details, please refer to the AWS Cognito documentation below:
https://docs.aws.amazon.com/cognito/latest/developerguide/logout-endpoint.html
In order to include needed parameters and work around this issue, you will need to include the parameters manually in the end session endpoint URL.
In order to do this the OIDC configuration in Qlik Cloud needs to be switched from automatic configuration to manual configuration, however as switching the checkbox does not clear the discovery URL and trigger an error at saving time, it will be needed to recreate the Identity Provider configuration in Qlik Cloud and choose "Manual Configuration" from the start.
1. Create a new Identity Provider configuration and copy over all settings
2. Select "Manual configuration" and copy manually the needed URLs from the OIDC configuration endpoint (the one ending with /.well-known/openid-configuration)
Copy over Authorization_endpoint, Issuer, JWKS URI, Token Endpoint. UserInfo endpoint is optional in the case you are using the IdP type "ADFS" to read groups from the ID token instead of the userinfo endpoint.
3. Now, also copy over the End Session endpoint but add the client_id and logout_uri parameters to the URL. You can use your Qlik Cloud tenant URL (URL-encoded) as the logout_uri. It should look like below:
https://xxxx.auth.ap-southeast-2.amazoncognito.com/logout?client_id=72h3dko46iqeivkgnjn009e5ju&logout_uri=https%3A%2F%2Fxiht01bhj1auga0.ap.qlikcloud.com
4. In this case, we have added https://xiht01bhj1auga0.ap.qlikcloud.com/ as the logout_uri parameter. so now we need to add this URL to allowed callback URLs and allowed sign-out URLs in AWS Cognito.
You should now be able to log out from Qlik Cloud without getting the error.
Qlik Sense allows for three settings that may influence the perceived connection and session timeout period. These are the "Session Inactivity Timeout", "Keep-Alive Timeout", and "TCP Websocket keep-alive" settings.
Note: Adjusting the below settings can help when working with slow internet connectivity or wanting to extend the session inactivity. However, session disconnect issues can be caused by other network connectivity issues and by system resource shortage as well and may require additional troubleshooting. See Hub access times out with: Error Connection lost. Make sure that Qlik Sense is running properly
This is the maximum timeout for a single HTTP request. The default value is 10 seconds. During the defined keep alive timeout value, the connection between end user and Qlik Sense will remain open.
It serves as protection against denial-of-service attacks. That is, if an ongoing request exceeds this period, Qlik Sense proxy will close the connection.
Increase this value if your users work over slow connections and experience closed connections for which no other workaround has been found. Make sure to take the mentioned DoS consideration above into account.
This is the browser authentication session time out ( 30 minutes by default set under Virtual Proxy in QMC ). This sets a cookie on the client machine with the name X-Qlik-Session. This cookie can be traced in Fiddler or Developer tools under the header tab.
If the session cookie header value does not get passed, is destroyed, or modified between the end user client and the Qlik Sense server while 'in-flight' the user session is terminated and the user is logged out.
By default, it will be destroyed after 30 minutes of inactivity or when the browser is closed.
This is another setting that may help keep the connection open in certain environments. See Enabling TCP Keep Alive Functionality In Qlik Sense. Note that customers who don't experience any issues with web sockets terminated by the network due to inactive SHOULD NOT switch this feature ON since it may potentially cause Qlik Sense to send unnecessary traffic on the network towards the client.