This article explains how to get started with the .NET SDK

The following must be installed to test the example provided in this article:

• Microsoft Visual Studio (The community version can be used)
• Qlik Sense Server

*It does not need to be installed on the same machine, but proper ports must be open for incoming connection on the Qlik Sense server side.

Please also make sure that the user connecting from the .NET SDK has a license assigned

Environments:

Resolution:

1. Open Visual Studio and create a new project. In this example, we have selected "Console App (.NET Framework)"
2. Go to Tools > NuGet Package manager > Package Manager Console
3. Use "Install-Package QlikSense.NetSDK" to install the latest version of the SDK or "Install-Package QlikSense.NetSDK -Version x.x.x" to install a specific version (It must match the Qlik Sense server version). Go to https://www.nuget.org/packages/QlikSense.NetSDK/ to find the correct command for your version if you are unsure.
4. Once the package is installed, it will appear in the References for your project. We can create a small program with the following code to connect to the Qlik Sense server and show Product version, OS Name and OS Version.

          In the below program, we are using Windows authentication, so we have to use the AsNtlmUserViaProxy function to specify that we want to use Windows authentication.

// using System, etc. 
using Qlik;
using Qlik.Engine;

namespace ConsoleApp1
{
    class Program
    {
        static void Main(string[] args)
        {
            Uri uri = new Uri("https://qlikserver3.domain.local/");
            ILocation location = Qlik.Engine.Location.FromUri(uri);
            location.AsNtlmUserViaProxy(certificateValidation: false);
            using (var hub = location.Hub())
            {
                Console.WriteLine("Product Version: " + hub.ProductVersion());
                Console.WriteLine("OS Name:    " + hub.OSName());
                Console.WriteLine("OS Version: " + hub.OSVersion());
                Console.WriteLine("Press enter to close...");
                Console.ReadLine();
            }
        }
    }
}

• This should show something like this:

Product Version: 3.2.3
OS Name: WindowsNT
OS Version: 6.2.9200
Press enter to close...

Connect to an app:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Qlik;
using Qlik.Engine;

namespace ConsoleApp1
{
    class Program
    {
        static void Main(string[] args)
        {
            Uri uri = new Uri("https://qlikserver1.domain.local/");
            ILocation location = Qlik.Engine.Location.FromUri(uri);
            location.AsNtlmUserViaProxy(certificateValidation: false);
            IAppIdentifier appIdentifier = location.AppWithId("03bbedf2-2c45-4e10-bf5a-fded0c21cda8");
            using (var app = location.App(appIdentifier))
            {
                var layout = app.GetAppLayout();
                Console.WriteLine(layout.Title + " [" + app.Type + "]");
                Console.ReadLine();
            }
        }
    }
}

Result:
Test1 [Doc]


Tip: If the program does not run, try to build it and execute the .exe file from the Qlik Sense server. If that works locally from the server itself, it might be that there is a network device that is not allowing the connection or the proper ports are not open.

During the upgrade to Talend Studio 8.0.1, opening any Job is not possible due to an error message:

java.lang.NoClassDefFoundError: org/talend/designer/runprocess/maven/MavenJavaProcessor.

Resolution

Force the Java locales to be English by adding the following entries to the Studio ini file:

-Duser.country=US
-Duser.language=en

Cause

You can catch error messages such as: 

java.io.FileNotFoundException: Repository not found: 

https://update.talend.com/Studio/8/updates/latest/messages_sv_SE.properties

java.io.FileNotFoundException: Repository not found: 

https://update.talend.com/Studio/8/updates/latest/messages_gr_GR.properties

The Studio is set to use unsupported locales.

Environment

• Talend Data Integration 8.0.1

In this article, we walk you through the requirements and process of how to upgrade and unbundle an existing Qlik Sense Repository Database (see supported scenarios) as well as how to install a brand new Repository based on PostgreSQL. We will use the Qlik PostgreSQL Installer (QPI).

For a manual method, see How to manually upgrade the bundled Qlik Sense PostgreSQL version to 12.5 version.

Using the Qlik Postgres Installer not only upgrades PostgreSQL; it also unbundles PostgreSQL from your Qlik Sense Enterprise on Windows install. This allows for direct control of your PostgreSQL instance and facilitates maintenance without a dependency on Qlik Sense. Further Database upgrades can then be performed independently and in accordance with your corporate security policy when needed, as long as you remain within the supported PostgreSQL versions. See How To Upgrade Standalone PostgreSQL.

Index

• Supported Scenarios
• Upgrades
• New installs
• Requirements
• Known limitations
• Installing anew Qlik Sense Repository Database using PostgreSQL
• Qlik PostgreSQL Installer - Download Link
• Upgrading an existing Qlik Sense Repository Database
• The Upgrade
• Next Steps and Compatibility with PostgreSQL installers
• How do I upgrade PostgreSQL from here on?
• Troubleshooting and FAQ
• Related Content

Video Walkthrough

Video chapters:

• 01:02 - Intro to PostgreSQL Repository
• 02:51 – Prerequisites
• 03:24 - What is the QPI tool?
• 05:09 - Using the QPI tool
• 09:27 - Removing the old Database Service
• 11:27 - Upgrading a stand-alone to the latest release
• 13:39 - How to roll-back to the previous version
• 14:46 - Troubleshooting upgrading a patched version
• 18:25 - Troubleshooting upgrade security error
• 21:15 - Additional config file settings

Supported Scenarios

Upgrades

The following versions have been tested and verified to work with QPI (1.4.0): 

Qlik Sense February 2022 to Qlik Sense November 2023.

If you are on a Qlik Sense version prior to these, upgrade to at least February 2022 before you begin.  

Qlik Sense November 2022 and later do not support 9.6, and a warning will be displayed during the upgrade. From Qlik Sense August 2023 a upgrade with a 9.6 database is blocked.

New installs

The Qlik PostgreSQL Installer supports installing a new standalone PostgreSQL database with the configurations required for connecting to a Qlik Sense server. This allows setting up a new environment or migrating an existing database to a separate host.

Requirements

• Review the QPI Release Notes before you continue

• Using the Qlik PostgreSQL Installer on a patched Qlik Sense version can lead to unexpected results. If you have a patch installed, either:

  • Uninstall all patches before using QPI (see Installing and Uninstalling Qlik Sense Patches) or
  • Upgrade to an IR release of Qlik Sense which supports QPI

• The PostgreSQL Installer can only upgrade bundled PostgreSQL database listening on the default port 4432.
• The user who runs the installer must be an administrator.
• The backup destination must have sufficient free disk space to dump the existing database  
• The backup destination must not be a network path or virtual storage folder. It is recommended the backup is stored on the main drive. 
• There will be downtime during this operation, please plan accordingly
• If upgrading to PostgreSQL 14 and later, the Windows OS must be at least Server 2016

Known limitations

• Cannot migrate a 14.8 embedded database to a standalone
• Using QPI to upgrade a standalone database or a database previously unbundled with QPI is not supported.
• The installer itself does not provide an automatic rollback feature.

Installing a new Qlik Sense Repository Database using PostgreSQL

1. Run the Qlik PostgreSQL Installer as an administrator 
   
   
2. Click on Install 
   
   
3. Accept the Qlik Customer Agreement 
   
   
4. Set your Local database settings and click Next. You will use these details to connect other nodes to the same cluster. 
   
   
5. Set your Database superuser password and click Next
   
   
6. Set the database installation folder, default: C:\Program Files\PostgreSQL\14

   Do not use the standard Qlik Sense folders, such as C:\Program Files\Qlik\Sense\Repository\PostgreSQL\ and C:\Programdata\Qlik\Sense\Repository\PostgreSQL\.

7. Set the database data folder, default: C:\Program Files\PostgreSQL\14\data

   Do not  use the standard Qlik Sense folders, such as C:\Program Files\Qlik\Sense\Repository\PostgreSQL\ and C:\Programdata\Qlik\Sense\Repository\PostgreSQL\.

8. Review your settings and click Install, then click Finish
   
   
9. Start installing Qlik Sense Enterprise Client Managed. Choose Join Cluster option.
   
   The Qlik PostgreSQL Installer has already seeded the databases for you and has created the users and permissions. No further configuration is needed.
   
   
10. The tool will display information on the actions being performed. Once installation is finished, you can close the installer.  
    
    If you are migrating your existing databases to a new host, please remember to reconfigure your nodes to connect to the correct host. How to configure Qlik Sense to use a dedicated PostgreSQL database 
    

Qlik PostgreSQL Installer - Download Link

Download the installer here.

Qlik PostgreSQL installer Release Notes

Upgrading an existing Qlik Sense Repository Database

The following versions have been tested and verified to work with QPI (1.4.0): 

February 2022 to November 2023.

If you are on any version prior to these, upgrade to at least February 2022 before you begin.  

Qlik Sense November 2022 and later do not support 9.6, and a warning will be displayed during the upgrade.  From Qlik Sense August 2023 a 9.6 update is blocked.

The Upgrade

1. Stop all services on rim nodes 
   
   
2. On your Central Node, stop all services except the Qlik Sense Repository Database 
   
   
   
   
3. Run the Qlik PostgreSQL Installer. An existing Database will be detected. 
   
   
   
   
4. Highlight the database and click Upgrade 
   
5. Read and confirm the (a) Installer Instructions as well as the Qlik Customer Agreement, then click (b) Next.
   
   
   
   
6. Provide your existing Database superuser password and click Next.
   
   
7. Define your Database backup path and click Next.
   
   
   
   
8. Define your Install Location (default is prefilled) and click Next.
   
   
   
   
9. Define your database data path (default is prefilled) and click Next.
   
   
    
10. Review all properties and click Upgrade.
    
    The review screen lists the settings which will be migrated. No manual changes are required post-upgrade.
    
    
    
    
11. The upgrade is completed. Click Close.
     
12. Open the Windows Services Console and locate the Qlik Sense Enterprise on Windows services.
    
    You will find that the Qlik Sense Repository Database service has been set to manual. Do not change the startup method. 
    
    
    
    You will also find a new postgresql-x64-14 service. Do not rename this service. 
    
    
    

     

13. Start all services except the Qlik Sense Repository Database service. 
    
    
14. Start all services on your rim nodes.
    
    
15. Validate that all services and nodes are operating as expected. The original database folder in C:\ProgramData\Qlik\Sense\Repository\PostgreSQL\X.X_deprecated
    

     

16. Uninstall the old Qlik Sense Repository Database service.

    This step is required. Failing to remove the old service will lead the upgrade or patching issues.

    1. Open a Windows File Explorer and browse to C:\ProgramData\Package Cache
       
       
    2. From there, search for the appropriate msi file.
       
       
       
       If you were running 9.6 before the upgrade, search PostgreSQL.msi
       If you were running 12.5 before the upgrade, search PostgreSQL125.msi
       
       
    3. The msi will be revealed.
       
       
    4. Right-click the msi file and select uninstall from the menu.
       
       
       
       
    
    
17. Re-install the PostgreSQL binaries. This step is optional if Qlik Sense is immediately upgraded following the use of QPI. The Sense upgrade will install the correct binaries automatically. 
    

    Failing to reinstall the binaries will lead to errors when executing any number of service configuration scripts. 

    If you do not immediately upgrade:
    
    
    1. Open a Windows File Explorer and browse to C:\ProgramData\Package Cache
       
       
    2. From there, search for the .msi file appropriate for your currently installed Qlik Sense version
       
       
       
       For Qlik Sense August 2023 and later: PostgreSQL14.msi
       Qlik Sense February 2022 to May 2023: PostgreSQL125.msi
       
       
    3. Right-click the file
    4. Click Open file location
       
       
       
       
    5. Highlight the file path, right-click on the path, and click Copy
       
       
       
       
    6. Open a Windows Command prompt as administrator
       
       
    7. Navigate to the location of the folder you copied
       
       Example command line: 
       
       cd C:\ProgramData\Package Cache\{GUID}
       
       Where GUID is the value of the folder name.
       
       
    8. Run the following command depending on the version you have installed: 
       
       Qlik Sense August 2023 and later
       
       msiexec.exe /qb /i "PostgreSQL14.msi" SKIPINSTALLDBSERVICE="1" INSTALLDIR="C:\Program Files\Qlik\Sense"
       
       Qlik Sense February 2022 to May 2023
       
       msiexec.exe /qb /i "PostgreSQL125.msi" SKIPINSTALLDBSERVICE="1" INSTALLDIR="C:\Program Files\Qlik\Sense"
       
       This will re-install the binaries without installing a database. If you installed with a custom directory adjust the INSTALLDIR parameter accordingly. E.g. you installed in D:\Qlik\Sense then the parameter would be INSTALLDIR="D:\Qlik\Sense".
       
       
18. Finalize the process by updating the references to the PostgreSQL binaries paths in the SetupDatabase.ps1 and Configure-Service.ps1 files. For detailed steps, see Cannot change the qliksenserepository password for microservices of the service dispatcher: The system cannot find the file specified.

If the upgrade was unsuccessful and you are missing data in the Qlik Management Console or elsewhere, contact Qlik Support.  

Next Steps and Compatibility with PostgreSQL installers

Now that your PostgreSQL instance is no longer connected to the Qlik Sense Enterprise on Windows services, all future updates of PostgreSQL are performed independently of Qlik Sense. This allows you to act in accordance with your corporate security policy when needed, as long as you remain within the supported PostgreSQL versions.

Your PostgreSQL database is fully compatible with the official PostgreSQL installers from https://www.enterprisedb.com/downloads/postgres-postgresql-downloads. 

How do I upgrade PostgreSQL from here on?

See How To Upgrade Standalone PostgreSQL, which documents the upgrade procedure for either a minor version upgrade (example: 14.5 to 14.8) or a major version upgrade (example: 12 to 14). Further information on PostgreSQL upgrades or updates can be obtained from Postgre directly.

Troubleshooting and FAQ

1. If the installation crashes, the server reboots unexpectedly during this process, or there is a power outage, the new database may not be in a serviceable state. Installation/upgrade logs are available in the location of your temporary files, for example:
   
   C:\Users\Username\AppData\Local\Temp\2
   
   A backup of the original database contents is available in your chosen location, or by default in:
   
   C:\ProgramData\Qlik\Sense\Repository\PostgreSQL\backup\X.X
   
   The original database data folder has been renamed to:
   
   C:\ProgramData\Qlik\Sense\Repository\PostgreSQL\X.X_deprecated
   
   
2. Upgrading Qlik Sense after upgrading PostgreSQL with the QPI tool fails with:
   
   This version of Qlik Sense requires a 'SenseServices' database for multi cloud capabilities. Ensure that you have created a 'SenseService' database in your cluster before upgrading. For more information see Installing and configuring PostgreSQL.
   
   See Qlik Sense Upgrade fails with: This version of Qlik Sense requires a _ database for _.
   
   To resolve this, start the postgresql-x64-XX service.

The information in this article is provided as-is and to be used at own discretion. Depending on tool(s) used, customization(s), and/or other factors ongoing support on the solution below may not be provided by Qlik Support. The video in this article was recorded in a earlier version of QPI, some screens might differ a little bit.

Related Content

Qlik PostgreSQL installer version 1.3.0 Release Notes  
Techspert Talks - Upgrading PostgreSQL Repository Troubleshooting
Backup and Restore Qlik Sense Enterprise documentation
Migrating Like a Boss
Optimizing Performance for Qlik Sense Enterprise
Qlik Sense Enterprise on Windows: How To Upgrade Standalone PostgreSQL
How-to reset forgotten PostgreSQL password in Qlik Sense
How to configure Qlik Sense to use a dedicated PostgreSQL database
Troubleshooting Qlik Sense Upgrades 

These two errors below may be found in the technical.log.

1. ERROR TalendRuntimeServerStateChecker - init talendRuntime Error!Failed to retrieve
   RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException:
   Connection refused to host: xxxx; nested exception is: java.net.ConnectException:Connection is refused] 
2. WARN TalendRuntimeServerStateChecker - TalendRuntime's AdminConsole is not running

When checking the server in the Talend Administration Center conductor section, the Status server is: DOWN

Resolution

In order to fix the first error below:

ERROR TalendRuntimeServerStateChecker - init talendRuntime Error!Failed to retrieve
RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException:
Connection refused to host: xxxx; nested exception is: java.net.ConnectException:
Connection is refused]

Please add the RMI server hostname to xxxx-wrapper.conf file

1. Stop Talend Runtime and Talend Administration Center.
2. Go to <Talend Runtime Installation Folder>/etc folder.
3. Open the <Talend Runtime Installation Folder>/etc/xxxx-wrapper.conf file.
4. Try adding 
   

   -Djava.rmi.server.hostname=<Enter hostname of Runtime here>

    

   to the "runtime/etc/xxxx-wrapper.conf" file as below:
   

   wrapper.java.additional.nn=-Djava.rmi.server.hostname=<Enter hostname of Runtime here>

   
   For example: 

   wrapper.java.additional.15=-Djava.rmi.server.hostname= <Enter hostname of Runtime here>

5. Save the file.
6. Start both Talend Runtime and Talend Administration Center. 

In order to fix the error below:

WARN TalendRuntimeServerStateChecker - TalendRuntime's AdminConsole is not running

Install the web console feature in Talend Runtime.

1. Login to Talend Runtime using a command tool.
2. Execute the following command:

feature:install webconsole

Cause

When facing the error below:

ERROR TalendRuntimeServerStateChecker - init talendRuntime Error!Failed to retrieve
RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException:
Connection refused to host: xxxx; nested exception is: java.net.ConnectException:
Connection is refused]

This may be because the RMI server hostname is not recognized in Talend Runtime.

When facing the error below:

WARN TalendRuntimeServerStateChecker - TalendRuntime's AdminConsole is not running

This may be because the web console is not running.

Environment

Talend Administration Center 

After following the Qlik article Configure Qlik Sense to use a dedicated PostgreSQL server and update the connection string to point to the new host, Qlik Sense appears to work fine at first. However, access by the Qlik Sense Service account to the old host is registered or errors are found in the logs similar to what is found in the example below:

In the AppDistributionService trace logs:

307308 20210104T170444.859+01:00 INFO QLIKSERVER 70 DOMAIN\serviceaccountname Retry attempt: 10/10. Previous result: "No such host is known". 304796
307309 20210104T170451.812+01:00 ERROR QLIKSERVER 70 DOMAIN\serviceaccountname Error processing message queue notifications. System.Net.Sockets.SocketException (11001): No such host is known
at System.Net.Dns.HostResolutionEndHelper(IAsyncResult asyncResult)

Resolution

There are a few new services starting on the September 2019 release of Qlik Sense which did not exist in June 2018. Some of them were also removed/replaced on the new release.

Option 1 (recommended): Updating the files using PowerShell:

1. Stop all the Qlik Sense services
2. Open a PowerShell command prompt in administrator mode and then, either run the following:
   
   

   # Set the Installation Directory for Qlik Sense
   $installDir = 'C:\Program Files\Qlik\Sense\'
   #change that to the new database server
   $newdatabasehostname='new database server'
   # Specify the new password for the qliksenserepository account
   $password = 'MyNewPassword'
   # Find all Configure-Service.ps1 scripts in the installation directory and execute them
   $files = Get-ChildItem -Path $installDir -Include Configure-Service.ps1 -Recurse
   foreach ($file in $files) {
       $ScriptToRun=$($file.FullName)
       &$ScriptToRun $newdatabasehostname 4432 qliksenserepository $password -postgresHome 'D:\Qlik\Sense\Repository\PostgreSQL\12.5'
   }​

   
   In case of having a custom path Qlik Sense installation, change the last line to:

&$ScriptToRun $newdatabasehostname 4432 qliksenserepository $password -postgresHome 'D:\Qlik\Sense\Repository\PostgreSQL\12.5'
}

Where "D:\Qlik\Sense" is Qlik Sense installed folder and where "12.5" is the Postgres version installed.

Option 2: Manually update each file

Change to the following directories with the commands below before running the Configure-Service.ps1 command listed further below.

Repeat the step for every subfolder mentioned below.

Note: If you previously upgraded you may find folders in your system which are not listed here. Ignore them. No steps need to be taken.

Qlik Sense February 2021 and later.

Locate Configure-Service.ps1 in C:\program files\Qlik\Sense\.

It lists all services that need to be changed.

Qlik Sense June 2018:

Qlik Sense September 2019:

Example:

For AppDistributionService...

PS C:\> cd 'C:\Program Files\Qlik\Sense\AppDistributionService'
PS C:\Program Files\Qlik\Sense\AppDistributionService> .\Configure-Service.ps1 DatabaseHost Databaseport DatabaseUser DatabasePassword

e.g. .\Configure-Service.ps1 newdatabasehostname 4432 qliksenserepository Password123!

The answer will look similar to this:

AppDistributionService configuration started.
WARNING: Skiping the database initialization. No superuser or password specified.
Reading the settings file.
Saving the modified settings.
Exporting the copy of the invocation parameters.
AppDistributionService configuration successful.

Related Content:

• Configure Qlik Sense to use a dedicated PostgreSQL server

Affected versions:

• Qlik Sense Enterprise on Windows all versions, including November 2024

CVE-2024-7348 is a Time-of-Check Time-of-Use (TOCTOU) race condition vulnerability in pg_dump, a utility used for backing up PostgreSQL databases. This vulnerability allows an attacker to replace a relation type (such as a table or sequence) with a view or foreign table right when pg_dump is running. Because pg_dump often runs with superuser privileges, this attack could execute arbitrary SQL code, leading to unauthorized actions or data corruption. Source: https://www.postgresql.org/support/security/CVE-2024-7348/

How does this vulnerability impact Qlik Sense Enterprise on Windows?

Qlik can confirm that pg_dump is not actively used in the Qlik Sense Enterprise on Windows code.

How to mitigate the vulnerability 

Both steps are required to fully mitigate the issue.

• Upgrade PostgreSQL. This requires a standalone instance of PostgreSQL. See Upgrading and unbundling the Qlik Sense Repository Database using the Qlik PostgreSQL Installer on how to unbundle PostgreSQL if necessary.

  Always verify compatibility between your Qlik Sense version and PostgreSQL before planning an upgrade.

• Delete the pg_dump.exe located in the default Qlik Sense Enterprise on Windows Postgresql install folder: C:\Program Files\Qlik\Sense\Repository\Postgresql\14\
  
  The updated pg_dump.exe in, for example, C:\program files\postgresql\14 does not need to be removed. 

Does the Qlik PostgreSQL Installer (QPI) use the pgdump.exe?

QPI does not utilise pgdump.exe.

After upgrading to Qlik Sense Enterprise on Windows May 2023 (or later), the Qlik Sense Repository Service may cause CPU usage spikes on the central node. In addition, the central Engine node may show an increased average RAM consumption while a high volume of reloads is being executed. 

The Qlik Sense System_Repository log file will read:

"API call to Engine service was successful: The retrieved 'StaticByteSize' value for app 'App-GUID' is: 'Size in bytes' bytes." 

Default location of log: C:\ProgramData\Qlik\Sense\Log\Repository\Trace\[Server_Name]_System_Repository.txt

This activity is associated with the ability to see the Base memory size in the Qlik Sense Enterprise Management Console. See Show base memory size of apps in QMC.

Resolution

The feature to see Base memory size can be disabled. This may require an upgrade and will require downtime as configuration files need to be changed.

Take a backup of your environment before proceeding.

1. Verify the version you are using and verify that you are at least on one of the versions (and patches) listed: 
   
   
   • May 2023 Patch 13
   • August 2023 Patch 10
   • November 2023 Patch 4 
   • February 2024 IR
     
2. Back up the following files:
   
   
   • C:\Program Files\Qlik\Sense\CapabilityService\capabilities.json
   • C:\Program Files\Qlik\Sense\Repository\Repository.exe.config
     
     *where C:\Program Files can be replaced by your custom installation path.
     
     
3. Stop the Qlik Sense Repository Service and the Qlik Sense Dispatcher Service
   
4. Open and edit the capabilities.json with a text editor (using elevated permissions)
   1. Open C:\Program Files\Qlik\Sense\CapabilityService\capabilities.json
   2. Locate: "enabled": true,"flag": "QMCShowAppSize"
   3. Change this to: "enabled": false,"flag": "QMCShowAppSize"
   4. Save the file  
5. Open and edit the repository.exe.config with a text editor (using elevated permissions)
   1. Open C:\Program Files\Qlik\Sense\Repository\Repository.exe.config
   2. Locate: </appSettings>
   3. Before the </appSettings> closing statement, add:
      <add key="AppStaticByteSizeUpdate.Enable" value="false"/>
   4. Save
6. Start all Qlik Sense services
7. Verify you can access the Management Console

If any issues occur, revert the changes by restoring the backed up files and open a ticket with Support providing the changed versions of repository.exe.config, capabilities.json mentioning this article. 

Related Content

Show base memory size of apps in QMC

Internal Investigation:

QB-22795
QB-24301

Environment

Qlik Sense Enterprise on Windows  May 2023, August 2023, November 2023, February 2024. 

With content cache-controls, you can modify the cache behavior of the browser. The cache-control is used on endpoints handled by the repository service. This functionality is disabled by default and can be enabled by modifying the files Repository.exe.conf and capabilities.json, followed by a restart of the Qlik Sense Service Dispatcher and the Qlik Sense Repository Service.

Cache-control for endpoints handled by the repository service was introduced in Qlik Sense Enterprise on Windows February 2022 (Patch 1). In the May 2022 release further enhancements were made to extend caching policies for requests specific to the Hub service. Upgrade Qlik Sense to make use of the new features. 

In this article, we walk you through modifying the cache control policy and how to apply it globally across your Qlik Sense Enterprise on Windows environment. 

Before modifying the cache-controls

Please note the following before modifying cache control headers:

• It is generally not recommended to manually alter the cache-control headers.
• Make sure that you understand how the browser caching behavior is affected by the changes.

• Prior to applying any changes in a production environment, evaluate the risks of the changes.

Enable the feature

The feature is disabled by default.  For details, see Configuring content cache-control.

1. Stop all Qlik Sense Enterprise on Windows services
   
   
2. Open C:\Program Files\Qlik\Sense\Repository\Repository.exe.conf
   
   
3. Modify the following key and set it to true:
   
   <add key="ContentCacheControl" value="true" />
   
   Save the file.
   
   
4. Open C:\Program Files\Qlik\Sense\CapabilityService\capabilities.json
   
   
5. Add the following flag:
   
   {"contentHash":"2ae4a99c9f17ab76e1eeb27bc4211874","originalClassName":"FeatureToggle","flag":"QMC_CONTENT_CACHE_CONTROL","enabled":true}
   
   Save the file.
   
   
6. Start all Qlik Sense Services
   
   
7. After the initial startup, restart the Qlik Sense Dispatcher Service
   

   A second restart of the Qlik Sense Dispatcher Service is required to propagate the cache control settings correctly.

Modify cache-controls in the Qlik Sense Management Console

1. After enabling cache-control, open the Qlik Sense Management Console
2. Go to Content Libraries
   
   
   
   
3. Open the Content Library you wish to configure
   
   
   
   
4. In the Associated Items menu, select Content cache-controls
5. Click Link new content cache control
   
   
   
   
6. Configure the cache-controls as per Content cache-controls  and click Add
   
   The cache control will automatically be linked.
   
   
   
   
7. Restart all Qlik Sense Services
   
   
8. After the initial startup, restart the Qlik Sense Dispatcher Service
   

   A second restart of the Qlik Sense Dispatcher Service is required to propagate the cache control settings correctly.

These are the predefined cache-control header values:

"CachePolicyEnum": {
    "values": [
        "0: Normal", -> public, max-age=3600
        "1: MustRevalidate", -> public, must-revalidate, max-age=0
        "2: PrivateNormal", -> private, max-age=3600
        "3: PrivateMustRevalidate", -> private, must-revalidate, max-age=0
        "4: NoStore" -> no-store
    ]
}

To modify the cache-control policy on a global level

You can create a global cache control policy (not tied to a content library) by using either the API or an SQL query against the QRS database. A restart of the Qlik Sense Service Dispatcher and Qlik Sense Repository service is required after.

API example:

POST /qrs/ContentCacheControl

{
    "id": "e07ca071-7d72-417d-93c2-d60687f747a8",
    "name": "api",
    "contentLibrary": null,
    "filter": "api",
    "maxAge": 3600,
    "cachePolicy": 4,
    "privileges": null
}

Database Query example:

INSERT INTO public."ContentCacheControls" ("ID", "CreatedDate", "ModifiedDate", "ModifiedByUserName", "Name", "Filter", "MaxAge", "CachePolicy", "ContentLibrary_ID")
VALUES ('e07ca071-7d72-417d-93c2-d60687f747a8', '2022-08-19 14:28:48.077709', '2022-08-19 14:28:48.077709', 'RDLUND\svc-silver', 'api', 'api', 3600, 4, null);

• Please update the values of "CreatedDate", "ModifiedDate", "ModifiedByUserName" keys and keep the rest of them unchanged. The ID can be any unique quid value.
• Please modify the value of "Filter" to a customised filter by using regex as per Content cache-controls.

The information in this article is provided as-is and to be used at own discretion. Depending on tool(s) used, customization(s), and/or other factors ongoing support on the solution below may not be provided by Qlik Support.

Related Content 

What's new in Qlik Sense February 2022 
What's new in Qlik Sense May 2022 

Environment

Qlik Sense Enterprise on Windows 

Internal Investigation ID(s):

QB-11410

What is the default user session timeout for Qlik Sense Business and Qlik Sense Enterprise SaaS? Can the session timeout for Qlik Cloud be changed?

The default (fixed) value is set to 30 minutes. This is controlled by SESSION_TTL.

It is not currently possible to adjust the session timeouts in the Qlik Cloud.

Environment:

Qlik Cloud 

You find the following error message in the logs when accessing Git repository:

 WARN ProjectViewAdapter - <Git URL here>: not authorized
 org.eclipse.jgit.api.errors.TransportException: <Git URL here>:not authorized

Resolution

1. Check the following documentations if you have configured Git correctly.
   • Installing and configuring Git
   • Setting up Git parameters
2. Check if the expiration date for Git Access Token has expired.
3. Check if you have enough permissions set for the Git Access Token.

Cause

It may be related to authentication issues.

Related Content

If you are looking for how to enable two factor authentication in Talend by using personal access tokens, please take a look at this article:

Talend-Cloud-Two-Factor-Authentication-using-personal-access

Environment

Talend Cloud 

While updating the qliksenserepository user with any of the Configure-Service.ps1 scripts, you may encounter the following error: 

Invoke-Query : Exception calling "Start" with "0" argument(s): "The system cannot find the file specified"
At C:\Program Fi 1es\Qlik\Sense\Mobi1i tyRegistrarService\install\instal1-uti1s\Postgres.psl:56 char :10
+ $ret = Invoke-Query $command
+ Categorylnfo : NotSpecified: (:) [Invoke-Query], MethodlnvocationException
+ FullyQualifiedErrorld : Win32Exception,Invoke-Query

Cause

Some Power-shell scripts have not been updated for the new default value of the Database home.

Different versions of Qlik Sense Enterprise for Windows come with their own bundled PostgreSQL version.

Examples: Qlik Sense Enterprise for Windows from May 2021 to May 2023 will install PostgreSQL 12.5, while previous releases used PostgreSQL 9.6 as a default during a new install. Similarly, from Qlik Sense Enterprise on Windows August 2023 onwards, Qlik pre-installs PostgreSQL 14.8 as the default database. For more details on the default, see the System Requirements for your respective version.

When you upgrade from an earlier release you will keep 9.6 or 12.5 and you should not have the issue.

This leads to issues with the following line:

Variation One:

[string]$postgresHome = "$senseInstallPath\Repository\PostgreSQL\9.6",

Variation Two:

[string]$postgresHome = "$senseInstallPath\Repository\PostgreSQL\12.5",

Environment

• Qlik Sense Enterprise on Windows

Resolution

Apply the actions in this solution on all nodes.

Depending on your version of Qlik Sense Enterprise on Windows different corrections will be needed.

1. Search for all *.ps1 scripts in C:\Program Files\Qlik\Sense
   
   For example, you can use Notepad++, utilizing its Search > Find in Files option: 
   
   
   
   Your directory may vary if you have used an alternative folder.
   
   
2. Find out what version of PostgreSQL binaries you have installed with Qlik Sense Enterprise on Windows:
   
   
   1. Open a Windows File Explorer and browse to C:\Program Files\Qlik\Sense\Repository\PostgreSQL
      
      
   2. You will find the PostgreSQL binaries folder
      
      
      
      
3. After identifying your PostgreSQL binaries version, replace the old path reference in the Configure-Service.ps1 files with the new version.
   
   Examples:
   
   replace 
   
   $senseInstallPath\Repository\PostgreSQL\9.6
   
   with 
   
   $senseInstallPath\Repository\PostgreSQL\12.5
   
   or
   
   replace 
   
   $senseInstallPath\Repository\PostgreSQL\12.5
   
   with 
   
   $senseInstallPath\Repository\PostgreSQL\14
   
   
4. Repeat this action on the path C:\Programdata\Qlik and update C:\Programdata\Qlik\Sense\QSMQ\scripts\SetupDatabase.ps1 
   
   

This is an example list of .ps1 files needing to be edited in the Qlik Sense Enterprise for Windows August 2023 version:

• C:\Programdata\Qlik\Sense\QSMQ\scripts\SetupDatabase.ps1 
  
  Line 12: [string]$postgresHome = "$senseInstallPath\Repository\PostgreSQL\12.5",
  
  
• C:\Program Files\Qlik\Sense\AppDistributionService\Configure-Service.ps1
  
  Line 13: [string]$postgresHome = "$senseInstallPath\Repository\PostgreSQL\12.5",
  
  
• C:\Program Files\Qlik\Sense\DataPrepService\install\Configure-Service.ps1
  
  Line 13: [string]$postgresHome = "$senseInstallPath\Repository\PostgreSQL\12.5",
  
  
• C:\Program Files\Qlik\Sense\HybridDeploymentService\Configure-Service.ps1
  
  Line 13: [string]$postgresHome = "$senseInstallPath\Repository\PostgreSQL\12.5",
  
  
• C:\Program Files\Qlik\Sense\Licenses\Configure-Service.ps1
  
  Line 13: [string]$postgresHome = "$senseInstallPath\Repository\PostgreSQL\12.5",
  
  
• C:\Program Files\Qlik\Sense\MobilityRegistrarService\install\Configure-Service.ps1
  
  Line 13: [string]$postgresHome = "$senseInstallPath\Repository\PostgreSQL\12.5",
  
  
• C:\Program Files\Qlik\Sense\NLAppSearch\install\Configure-Service.ps1
  
  Line 13: [string]$postgresHome = "$senseInstallPath\Repository\PostgreSQL\12.5",
  
  
• C:\Program Files\Qlik\Sense\NLBroker\install\Configure-Service.ps1
  
  Line 13: [string]$postgresHome = "$senseInstallPath\Repository\PostgreSQL\12.5",
  
  
• C:\Program Files\Qlik\Sense\NotifierService\install\Configure-Service.ps1
  
  Line 13: [string]$postgresHome = "$senseInstallPath\Repository\PostgreSQL\12.5",
  
  
• C:\Program Files\Qlik\Sense\PrecedentsService\install\Configure-Service.ps1
  
  Line 13: [string]$postgresHome = "$senseInstallPath\Repository\PostgreSQL\12.5",

This article describes how Qlik Application Automation can be used to perform and review app evaluations and take action accordingly.

Environment

• Qlik Application Automation
• Qlik Cloud Services

Example

In this example, we'll create an automation that evaluates an app before publishing it to a managed space.

Before you continue, create a new automation.

1. Search for the Qlik Cloud Services connector in the Block Library and add the Do Evaluation block to the automation. Configure an app ID by using the do lookup function. The Do Evaluation block will analyze the app and return metrics regarding the app's performance.
   
   

2. Add a Variable block to create a new variable of type list with the name vCacheOverrun. We will use this variable to store all objects from the evaluated app that take too long to cache.
   
   

3. The Do Evaluation block will analyze the app and return metrics regarding the app's performance. One metric we want to look at is the "ObjSlowCached" metric which contains slow objects.
   
   
4. We want to filter all objects in the ObjSlowCached list and add each item that takes too long to load to the list variable vCacheOverrun. To do this, add a Filter List block to the automation and use another variable block inside the loop from the Filter List block to add these items to the list variable. Configure the Filter List block to return all objects with a responseTimeSeconds greater than 0.3.
   
   
5. In the Variable block, add the object's id and responseTimeSeconds as a formatted string to the list.
   
   

6. Add a Condition block that verifies whether the vCacheOverrun variable is empty. If it's empty, it means there are no (too) slow cached objects and the app can be published. Add a Publish App To Managed Space block to the Yes outcome of the Condition block and add a Send Mail block from the Outlook connector to notify your team about the published app.

   

7. If the vCacheOverrun variable is not empty, the app should not be published. Add another Variable block vStringCache of type string to the No outcome of the Condition block. Set the value of this variable to the output from the vCacheOverrun variable and add the implode formula to transform the list variable into a string.

   
8. Add a Send Mail block from the Outlook connector to notify your team that the app wasn't published. Make sure to include the vStringCache variable in the body of the email.

Bonus

• Feel free to replace the Send Mail blocks in this automation with blocks from other messaging connectors like Microsoft Teams or Slack.

The information in this article is provided as-is and to be used at own discretion. Depending on tool(s) used, customization(s), and/or other factors ongoing support on the solution below may not be provided by Qlik Support.

Question

What Files Should I whitelist in my Antivirus Software for Talend Applications like Talend Administration Center and Job Server?

For Talend Administration Center, the folders that include jar files are recommended to be whitelisted.
For JobServer, the cache folder <JobServerInstallationFolder>/jobserver/agent/TalendJobServerFiles are recommended to be whitelisted.
The above are files that may have an impact on the performance. 

Windows Antivirus Software might cause file locks that can impact the cleanup of JobServer caches and temporary files created by jobs.
So it can be advisable to

• Make sure that jobs use a distinct temp folder that can also be excluded from scanning.
• The same apply for Talend Administration Center with generatedJobs folder and Logs folder.

 Also, as mentioned in the reference: Optimizing Talend Studio performance, the application installation folder is also recommended to whitelist for all applications, not just Talend Studio.

Environment

Talend Administration Center 

After installing TAC 8.0.1 and attempting to access the DB Config page in TAC, an error occurs:

500 The call failed on the server; see server log for details

The Oracle Database is utilized as TAC's database, and the compatible ODBC jar has been installed. The server log displays the primary error message as follows:

Error: caused by java.sql.SQLException: Non-supported character set (add orai18n.jar to your classpath).

Cause

The dependent orai18n.jar file is missing from the lib folder.

Resolution

Download the orai18n.jar file and ensure it is placed in both of the following folders:

<apache-tomcat>/lib

<apache-tomcat>/webapps/org.talend.administrator/WEB-INF/lib

Once completed, please restart the TAC server.

Environment

• Talend Administration Center 8.0.1

The Qlik Sense Engine allows for a Hard Max Limit to be set on memory consumption. 

To be able to enforce "Hard max limit" the engine requires access to a specific endpoint in the Windows operating system API.

If the operating system blocks calls to this endpoint, then an attempt to use the configuration "Hard max limit" will fail, and the engine will instead operate according to the "Soft max limit" configuration.

Refer to the documentation provided by Microsoft fur forther information: "SetProcessWorkingSetSizeEx"

How to review your memory settings in Qlik Sense or configure the Hard Max Limit:

1. Open the Qlik Sense Management Console in a supported browser
2. Navigate to Engines in the Management Console and select your engine
   
   
   
   
3. Expand the Advanced settings from the menu to the right
   
   
   
   
4. Review your memory settings and verify that Max Max Limit is selected
   
   

See Editing an engine - Qlik Sense for administrators for details.

To note:

Even with the hard limit set, it may still be possible for the host operating system to report memory spikes above the Max memory usage (%).

This is down to how the Qlik Sense Engine memory limit will be defined based on the total memory available.

Example:

• A server has 10 GB of memory in total, and the Max memory usage (%) is set to 90%.
• This will allow the process engine.exe to use 9 GB of memory. 
• Another application or process may at that point already be consuming 5 GB memory, and this will cause an overload of the system if the engine is set to use 9 GB. 
• The engine.exe will not be able to respect other services

The memory working setting limit is not a hard limit to set on the engine. This is a setting which determines how much we allocate and how far we are allowed to go before we start alarming on the working set beyond parameters. 

Internal Investigation IDs:

QLIK-96872, HLP-17517

If you have a web application server that is deployed on Talend Runtime, and need to change the timeout values, it is located in the file below: <RuntimeInstallationFolder>/etc/jetty.xml 

The following is a part of the code found in the jetty.xml file.
Please change the value for "http.timeout" in order to change the timeout value.

The default setting for the timeout value is 30000 milliseconds which equals to 30 seconds.

<Call name="addConnector">
　　<Arg>
　　　　<New class="org.eclipse.jetty.server.ServerConnector">
　　　　　　　<Set name="idleTimeout">
　　　　　　　　　<Property name="http.timeout" default="30000"/>
　　　　　　　</Set>
　　　　</New>
　　</Arg>
</Call>

Environment

Talend ESB 

Manually restart Qlik Sense services in right order is important because the Qlik Sense Repository Service is dependent on the Qlik Sense Repository Database (QRD) and the rest of the services are dependent on the Qlik Sense Repository Service.

Please review the Qlik Sense Online help for the relevant Sense version to find the Service Dependencies for your specific version. 

In a Multi-node environment, start the central node first and stop the rim nodes first.

Content:

• Correct Start Order:
• Central node
• Rim node
• Correct Stop Order:
• Central node
• Rim node
• Sample PS Script below for stopping and starting the Qlik Sense services:
• Optional monitoring of start process:

Correct Start Order:

Central node

1. Qlik Sense Repository Database (may also be called postgresql-x64-12)
2. (if installed) Qlik Sense Logging Service 
3. Qlik Sense Service Dispatcher
4. Qlik Sense Repository Service 
   Note: Delay for up to five seconds before starting the Qlik Sense Repository Service to allow the Dispatcher time to start its microservices. 
5. Qlik Sense Proxy Service, Qlik Sense Engine Service, Qlik Sense Scheduler Service, Qlik Sense Printing Service

Rim node

We are unable to provide a comprehensive start or stop order as rim nodes can have a number of different combinations of services installed. Please skip any services you do not have installed.

1. (if installed) Qlik Sense Logging Service 
2. Qlik Sense Service Dispatcher
3. Qlik Sense Repository Service 
   Note: Delay for up to five seconds before starting the Qlik Sense Repository Service to allow the Dispatcher time to start its microservices. 
4. Qlik Sense Proxy Service, Qlik Sense Engine Service, Qlik Sense Scheduler Service, Qlik Sense Printing Service

Correct Stop Order:

Central node

1. Qlik Sense Repository Database (QRD)
   This will first terminate all services with a dependency to the QRD (see Service dependencies on the Qlik Sense Online Help)
2. (if installed and not stopped already by step 1.) Qlik Sense Logging Service (If feature was enabled during installation)
3. Qlik Sense Service Dispatcher (QSD)

Rim node

We are unable to provide a comprehensive start or stop order as rim nodes can have a number of different combinations of services installed. Please skip any services you do not have installed.

1. Qlik Sense Proxy Service, Qlik Sense Engine Service, Qlik Sense Scheduler Service, Qlik Sense Printing Service
2. (if installed) Qlik Sense Logging Service 
3. Qlik Sense Repository Service
4. Qlik Sense Service Dispatcher (QSD)

Sample PS Script below for stopping and starting the Qlik Sense services:

Get-Service "Qlik*" | Where-Object {($_.Name -like "QlikSense*" -and $_.Name -notlike "QlikSenseRepositoryDatabase") -or ($_.Name -eq "QlikLoggingService")} | Stop-Service -Force 
Start-Service -Name "Qlik Sense Service Dispatcher" 
Start-Service -Name "QlikSenseServiceDispatcher" 
Start-Service "Qlik*"

Optional monitoring of start process:

Open the following location: C:\ProgramData\Qlik\Sense\Log\Repository\Trace
Monitor the folder after starting the Qlik Sense Repository Service (QRS). Files with appended time/date will disappear.
 

This article provides the necessary steps to back up an existing Qlik Enterprise Manager PostgreSQL database before restoring it to a new major version of PostgreSQL. We will use pgAdmin for the process.

For Qlik Enterprise Mangaer's System Requirements, including PostgreSQL compatibility, see Additional software requirements.

If you are upgrading within a major PostgreSQL release (example: 14.5 to 14.8), no manual backup and restore is necessary. Simply download the PostgreSQL installer and upgrade your current instance. 

If you are upgrading to a new major PostgreSQL release (example: 12.x to 14.x):

1. Backup the existing PostgreSQL database of your Qlk Enterprise Manager instance using pgAdmin:

   1. Right-click on the existing PostgreSQL database (in our example: postgres)

   2. Select Backup
      
      

   3. Fill in the required information:
      
      File Name: Your backup file name
      Format: TAR
      Role:  Current role name
      
      
   4. Click Backup
      
      
      
      
      
2. Download the new major version of PostgreSQL you want to upgrade to. For example, you may be moving from 12.x to 14.x.
3. Install the new PostgreSQL instance on the Qlik Enterprise Manager host
   
   for PostgreSQL-specific installation instructions, see PostgreSQL's manuals. 
   
   
4. In the new PostgreSQL instance, create a New User identical to the one used in the previously backed-up database 
   
   
5. In the new PostgreSQL instance, create a New Database with an identical name to the previously backed-up database
   
   
6. Restore your backup to the new database using pgAdmin:
   
   
   1. Right-click on the newly created database (in our example: postgres)
      
      
   2. Select Restore 
      
      
      
      
   3. Fill in the required information:
      
      Format: Custom or Tar
      File Name: Your backup file name
      Role: Previous role name
      
      
   4. Click Restore
      
      
      
      
7. Point Qlik Enterprise Manager to the new PostgreSQL database by following the steps in Unable to start collector in analytics module of Qlik Enterprise Manager when changing the database
   
   
8. Verify you can see the old analytics information

Environment

• Qlik Enterprise Manager