This article explains how to turn on PostgreSQL logs in Qlik Sense.

Environment:

Qlik Sense Enterprise on Windows 

Resolution:

1. On the Qlik Sense Server, open Postgresql.conf in C:\ProgramData\Qlik\Sense\Repository\PostgreSQL\9.6
2. Add or uncomment the following lines (refer to example of configuration file attached to this article):

logging_collector = on
log_directory = 'pg_log'
log_filename = 'postgresql-%Y-%m-%d_%H%M%S.log'
log_truncate_on_rotation = on
log_rotation_age = 1d    
client_min_messages = notice
log_min_messages = warning
log_min_error_statement = warning
log_line_prefix = '%t'

      3. Restart Qlik Sense Repository Database (This will also restart all other Qlik Sense services)
      4. Logs will be created in C:\ProgramData\Qlik\Sense\Repository\PostgreSQL\9.6\pg_log

If concerned about resource or performance consequences of enabling PostgreSQL logging, the following results have been validated during load testing by Qlik R&D:

Average response time:

Slightly higher with logging enabled but the biggest difference was only 4ms. Average time without logging 163ms, with logging 166ms.

Processor time:

PostgreSQL used very little CPU time. Average time without logging was 0.29%, with logging 0.33%.

Memory usage:

Here, slightly less memory was used with logging enabled. Average without logging 1.16GB, with logging 1.13GB.

Log files on disk:

With logging enabled, a number of files were created on the local disk:   About 10MB of data were created every minute.

In this article, we walk you through the requirements and process of how to upgrade and unbundle an existing Qlik Sense Repository Database (see supported scenarios) as well as how to install a brand new Repository based on PostgreSQL. We will use the Qlik PostgreSQL Installer (QPI).

For a manual method, see How to manually upgrade the bundled Qlik Sense PostgreSQL version to 12.5 version.

Using the Qlik Postgres Installer not only upgrades PostgreSQL; it also unbundles PostgreSQL from your Qlik Sense Enterprise on Windows install. This allows for direct control of your PostgreSQL instance and facilitates maintenance without a dependency on Qlik Sense. Further Database upgrades can then be performed independently and in accordance with your corporate security policy when needed, as long as you remain within the supported PostgreSQL versions. See How To Upgrade Standalone PostgreSQL.

Index

• Supported Scenarios
• Upgrades
• New installs
• Requirements
• Known limitations
• Installing anew Qlik Sense Repository Database using PostgreSQL
• Qlik PostgreSQL Installer - Download Link
• Upgrading an existing Qlik Sense Repository Database
• The Upgrade
• Next Steps and Compatibility with PostgreSQL installers
• How do I upgrade PostgreSQL from here on?
• Troubleshooting and FAQ
• Related Content

Video Walkthrough

Video chapters:

• 01:02 - Intro to PostgreSQL Repository
• 02:51 – Prerequisites
• 03:24 - What is the QPI tool?
• 05:09 - Using the QPI tool
• 09:27 - Removing the old Database Service
• 11:27 - Upgrading a stand-alone to the latest release
• 13:39 - How to roll-back to the previous version
• 14:46 - Troubleshooting upgrading a patched version
• 18:25 - Troubleshooting upgrade security error
• 21:15 - Additional config file settings

Supported Scenarios

Upgrades

The following versions have been tested and verified to work with QPI (1.4.0): 

Qlik Sense February 2022 to Qlik Sense November 2023.

If you are on a Qlik Sense version prior to these, upgrade to at least February 2022 before you begin.  

Qlik Sense November 2022 and later do not support 9.6, and a warning will be displayed during the upgrade. From Qlik Sense August 2023 a upgrade with a 9.6 database is blocked.

New installs

The Qlik PostgreSQL Installer supports installing a new standalone PostgreSQL database with the configurations required for connecting to a Qlik Sense server. This allows setting up a new environment or migrating an existing database to a separate host.

Requirements

• Review the QPI Release Notes before you continue

• Using the Qlik PostgreSQL Installer on a patched Qlik Sense version can lead to unexpected results. If you have a patch installed, either:

  • Uninstall all patches before using QPI (see Installing and Uninstalling Qlik Sense Patches) or
  • Upgrade to an IR release of Qlik Sense which supports QPI

• The PostgreSQL Installer can only upgrade bundled PostgreSQL database listening on the default port 4432.
• The user who runs the installer must be an administrator.
• The backup destination must have sufficient free disk space to dump the existing database  
• The backup destination must not be a network path or virtual storage folder. It is recommended the backup is stored on the main drive. 
• There will be downtime during this operation, please plan accordingly
• If upgrading to PostgreSQL 14 and later, the Windows OS must be at least Server 2016

Known limitations

• Cannot migrate a 14.8 embedded database to a standalone
• Using QPI to upgrade a standalone database or a database previously unbundled with QPI is not supported.
• The installer itself does not provide an automatic rollback feature.

Installing a new Qlik Sense Repository Database using PostgreSQL

1. Run the Qlik PostgreSQL Installer as an administrator 
   
   
2. Click on Install 
   
   
3. Accept the Qlik Customer Agreement 
   
   
4. Set your Local database settings and click Next. You will use these details to connect other nodes to the same cluster. 
   
   
5. Set your Database superuser password and click Next
   
   
6. Set the database installation folder, default: C:\Program Files\PostgreSQL\14

   Do not use the standard Qlik Sense folders, such as C:\Program Files\Qlik\Sense\Repository\PostgreSQL\ and C:\Programdata\Qlik\Sense\Repository\PostgreSQL\.

7. Set the database data folder, default: C:\Program Files\PostgreSQL\14\data

   Do not  use the standard Qlik Sense folders, such as C:\Program Files\Qlik\Sense\Repository\PostgreSQL\ and C:\Programdata\Qlik\Sense\Repository\PostgreSQL\.

8. Review your settings and click Install, then click Finish
   
   
9. Start installing Qlik Sense Enterprise Client Managed. Choose Join Cluster option.
   
   The Qlik PostgreSQL Installer has already seeded the databases for you and has created the users and permissions. No further configuration is needed.
   
   
10. The tool will display information on the actions being performed. Once installation is finished, you can close the installer.  
    
    If you are migrating your existing databases to a new host, please remember to reconfigure your nodes to connect to the correct host. How to configure Qlik Sense to use a dedicated PostgreSQL database 
    

Qlik PostgreSQL Installer - Download Link

Download the installer here.

Qlik PostgreSQL installer Release Notes

Upgrading an existing Qlik Sense Repository Database

The following versions have been tested and verified to work with QPI (1.4.0): 

February 2022 to November 2023.

If you are on any version prior to these, upgrade to at least February 2022 before you begin.  

Qlik Sense November 2022 and later do not support 9.6, and a warning will be displayed during the upgrade.  From Qlik Sense August 2023 a 9.6 update is blocked.

The Upgrade

1. Stop all services on rim nodes 
   
   
2. On your Central Node, stop all services except the Qlik Sense Repository Database 
   
   
   
   
3. Run the Qlik PostgreSQL Installer. An existing Database will be detected. 
   
   
   
   
4. Highlight the database and click Upgrade 
   
5. Read and confirm the (a) Installer Instructions as well as the Qlik Customer Agreement, then click (b) Next.
   
   
   
   
6. Provide your existing Database superuser password and click Next.
   
   
7. Define your Database backup path and click Next.
   
   
   
   
8. Define your Install Location (default is prefilled) and click Next.
   
   
   
   
9. Define your database data path (default is prefilled) and click Next.
   
   
    
10. Review all properties and click Upgrade.
    
    The review screen lists the settings which will be migrated. No manual changes are required post-upgrade.
    
    
    
    
11. The upgrade is completed. Click Close.
     
12. Open the Windows Services Console and locate the Qlik Sense Enterprise on Windows services.
    
    You will find that the Qlik Sense Repository Database service has been set to manual. Do not change the startup method. 
    
    
    
    You will also find a new postgresql-x64-14 service. Do not rename this service. 
    
    
    

     

13. Start all services except the Qlik Sense Repository Database service. 
    
    
14. Start all services on your rim nodes.
    
    
15. Validate that all services and nodes are operating as expected. The original database folder in C:\ProgramData\Qlik\Sense\Repository\PostgreSQL\X.X_deprecated
    

     

16. Uninstall the old Qlik Sense Repository Database service.

    This step is required. Failing to remove the old service will lead the upgrade or patching issues.

    1. Open a Windows File Explorer and browse to C:\ProgramData\Package Cache
       
       
    2. From there, search for the appropriate msi file.
       
       
       
       If you were running 9.6 before the upgrade, search PostgreSQL.msi
       If you were running 12.5 before the upgrade, search PostgreSQL125.msi
       
       
    3. The msi will be revealed.
       
       
    4. Right-click the msi file and select uninstall from the menu.
       
       
       
       
    
    
17. Re-install the PostgreSQL binaries. This step is optional if Qlik Sense is immediately upgraded following the use of QPI. The Sense upgrade will install the correct binaries automatically. 
    

    Failing to reinstall the binaries will lead to errors when executing any number of service configuration scripts. 

    If you do not immediately upgrade:
    
    
    1. Open a Windows File Explorer and browse to C:\ProgramData\Package Cache
       
       
    2. From there, search for the .msi file appropriate for your currently installed Qlik Sense version
       
       
       
       For Qlik Sense August 2023 and later: PostgreSQL14.msi
       Qlik Sense February 2022 to May 2023: PostgreSQL125.msi
       
       
    3. Right-click the file
    4. Click Open file location
       
       
       
       
    5. Highlight the file path, right-click on the path, and click Copy
       
       
       
       
    6. Open a Windows Command prompt as administrator
       
       
    7. Navigate to the location of the folder you copied
       
       Example command line: 
       
       cd C:\ProgramData\Package Cache\{GUID}
       
       Where GUID is the value of the folder name.
       
       
    8. Run the following command depending on the version you have installed: 
       
       Qlik Sense August 2023 and later
       
       msiexec.exe /qb /i "PostgreSQL14.msi" SKIPINSTALLDBSERVICE="1" INSTALLDIR="C:\Program Files\Qlik\Sense"
       
       Qlik Sense February 2022 to May 2023
       
       msiexec.exe /qb /i "PostgreSQL125.msi" SKIPINSTALLDBSERVICE="1" INSTALLDIR="C:\Program Files\Qlik\Sense"
       
       This will re-install the binaries without installing a database. If you installed with a custom directory adjust the INSTALLDIR parameter accordingly. E.g. you installed in D:\Qlik\Sense then the parameter would be INSTALLDIR="D:\Qlik\Sense".
       
       
18. Finalize the process by updating the references to the PostgreSQL binaries paths in the SetupDatabase.ps1 and Configure-Service.ps1 files. For detailed steps, see Cannot change the qliksenserepository password for microservices of the service dispatcher: The system cannot find the file specified.

If the upgrade was unsuccessful and you are missing data in the Qlik Management Console or elsewhere, contact Qlik Support.  

Next Steps and Compatibility with PostgreSQL installers

Now that your PostgreSQL instance is no longer connected to the Qlik Sense Enterprise on Windows services, all future updates of PostgreSQL are performed independently of Qlik Sense. This allows you to act in accordance with your corporate security policy when needed, as long as you remain within the supported PostgreSQL versions.

Your PostgreSQL database is fully compatible with the official PostgreSQL installers from https://www.enterprisedb.com/downloads/postgres-postgresql-downloads. 

How do I upgrade PostgreSQL from here on?

See How To Upgrade Standalone PostgreSQL, which documents the upgrade procedure for either a minor version upgrade (example: 14.5 to 14.8) or a major version upgrade (example: 12 to 14). Further information on PostgreSQL upgrades or updates can be obtained from Postgre directly.

Troubleshooting and FAQ

1. If the installation crashes, the server reboots unexpectedly during this process, or there is a power outage, the new database may not be in a serviceable state. Installation/upgrade logs are available in the location of your temporary files, for example:
   
   C:\Users\Username\AppData\Local\Temp\2
   
   A backup of the original database contents is available in your chosen location, or by default in:
   
   C:\ProgramData\Qlik\Sense\Repository\PostgreSQL\backup\X.X
   
   The original database data folder has been renamed to:
   
   C:\ProgramData\Qlik\Sense\Repository\PostgreSQL\X.X_deprecated
   
   
2. Upgrading Qlik Sense after upgrading PostgreSQL with the QPI tool fails with:
   
   This version of Qlik Sense requires a 'SenseServices' database for multi cloud capabilities. Ensure that you have created a 'SenseService' database in your cluster before upgrading. For more information see Installing and configuring PostgreSQL.
   
   See Qlik Sense Upgrade fails with: This version of Qlik Sense requires a _ database for _.
   
   To resolve this, start the postgresql-x64-XX service.

The information in this article is provided as-is and to be used at own discretion. Depending on tool(s) used, customization(s), and/or other factors ongoing support on the solution below may not be provided by Qlik Support. The video in this article was recorded in a earlier version of QPI, some screens might differ a little bit.

Related Content

Qlik PostgreSQL installer version 1.3.0 Release Notes  
Techspert Talks - Upgrading PostgreSQL Repository Troubleshooting
Backup and Restore Qlik Sense Enterprise documentation
Migrating Like a Boss
Optimizing Performance for Qlik Sense Enterprise
Qlik Sense Enterprise on Windows: How To Upgrade Standalone PostgreSQL
How-to reset forgotten PostgreSQL password in Qlik Sense
How to configure Qlik Sense to use a dedicated PostgreSQL database
Troubleshooting Qlik Sense Upgrades 

After following the Qlik article Configure Qlik Sense to use a dedicated PostgreSQL server and update the connection string to point to the new host, Qlik Sense appears to work fine at first. However, access by the Qlik Sense Service account to the old host is registered or errors are found in the logs similar to what is found in the example below:

In the AppDistributionService trace logs:

307308 20210104T170444.859+01:00 INFO QLIKSERVER 70 DOMAIN\serviceaccountname Retry attempt: 10/10. Previous result: "No such host is known". 304796
307309 20210104T170451.812+01:00 ERROR QLIKSERVER 70 DOMAIN\serviceaccountname Error processing message queue notifications. System.Net.Sockets.SocketException (11001): No such host is known
at System.Net.Dns.HostResolutionEndHelper(IAsyncResult asyncResult)

Resolution

There are a few new services starting on the September 2019 release of Qlik Sense which did not exist in June 2018. Some of them were also removed/replaced on the new release.

Option 1 (recommended): Updating the files using PowerShell:

1. Stop all the Qlik Sense services
2. Open a PowerShell command prompt in administrator mode and then, either run the following:
   
   

   # Set the Installation Directory for Qlik Sense
   $installDir = 'C:\Program Files\Qlik\Sense\'
   #change that to the new database server
   $newdatabasehostname='new database server'
   # Specify the new password for the qliksenserepository account
   $password = 'MyNewPassword'
   # Find all Configure-Service.ps1 scripts in the installation directory and execute them
   $files = Get-ChildItem -Path $installDir -Include Configure-Service.ps1 -Recurse
   foreach ($file in $files) {
       $ScriptToRun=$($file.FullName)
       &$ScriptToRun $newdatabasehostname 4432 qliksenserepository $password -postgresHome 'D:\Qlik\Sense\Repository\PostgreSQL\12.5'
   }​

   
   In case of having a custom path Qlik Sense installation, change the last line to:

&$ScriptToRun $newdatabasehostname 4432 qliksenserepository $password -postgresHome 'D:\Qlik\Sense\Repository\PostgreSQL\12.5'
}

Where "D:\Qlik\Sense" is Qlik Sense installed folder and where "12.5" is the Postgres version installed.

Option 2: Manually update each file

Change to the following directories with the commands below before running the Configure-Service.ps1 command listed further below.

Repeat the step for every subfolder mentioned below.

Note: If you previously upgraded you may find folders in your system which are not listed here. Ignore them. No steps need to be taken.

Qlik Sense February 2021 and later.

Locate Configure-Service.ps1 in C:\program files\Qlik\Sense\.

It lists all services that need to be changed.

Qlik Sense June 2018:

Qlik Sense September 2019:

Example:

For AppDistributionService...

PS C:\> cd 'C:\Program Files\Qlik\Sense\AppDistributionService'
PS C:\Program Files\Qlik\Sense\AppDistributionService> .\Configure-Service.ps1 DatabaseHost Databaseport DatabaseUser DatabasePassword

e.g. .\Configure-Service.ps1 newdatabasehostname 4432 qliksenserepository Password123!

The answer will look similar to this:

AppDistributionService configuration started.
WARNING: Skiping the database initialization. No superuser or password specified.
Reading the settings file.
Saving the modified settings.
Exporting the copy of the invocation parameters.
AppDistributionService configuration successful.

Related Content:

• Configure Qlik Sense to use a dedicated PostgreSQL server

1. Take a backup of the app and data associated with the app:
   1. Export the existing Operations Monitor app via the QMC (Apps > More actions > Export). Backup to a safe location.
   2. Backup the QVD files in the Central Node Log directory (C:\ProgramData\Qlik\Sense\Log by default)
   3. Verify you have an Operations Monitor QVF file in the DefaultApps section of the Qlik installation directory (C:\ProgramData\Qlik\Sense\Repository\DefaultApps by default). Ask support for a copy for your version of Qlik Sense if you cannot locate the file.
   4. Write down any reload tasks associated with the app so that you can restore them later
   5. Write down the original owner of the app (should be the internal repository user, INTERNAL\sa_repository)
2. Remove the Operations Monitor App and associated data files:
   1. Delete the Operations Monitor app via the QMC (Apps > Delete). This removes both the app and associated reload tasks.
   2. Delete the QVD files associated with the Operations Monitor App (governance*.qvd) in the Central Node Log directory (C:\ProgramData\Qlik\Sense\Log by default)
3. Restart all Qlik Sense services.
4. Re-import the Operations Monitor App from the installation directory:
   1. Locate the default Operations Monitor app in the DefaultApps section of the Qlik installation directory (C:\ProgramData\Qlik\Sense\Repository\DefaultApps by default)
   2. Import the app via the QMC (Apps > Import)
   3. Change ownership of the app to the same user as before (should have been the internal repository user, INTERNAL\sa_repository)
   4. Recreate any reload tasks noted from step 1
5. Reload the app, and verify the app works as expected.

Replicate Change Processing User Interface

Qlik Replicate's User Interface comes with a set of monitoring tools to help us understand and break down what is happening to the task on all levels. In this article, we will focus on the incoming changes from the CDC (Change Data Capture) process and understand where the changes are and what happens to the changes along the way.

To monitor the incoming changes for the task, we would need to go to the Monitor tab, switch to the Change Processing tab, and click on Incoming Changes. 

From this main screen, we see five vertical bars.  Here the bars are separated by the following:

• Incoming Changes
• Accumulating - In Memory
• Accumulating - On Disk
• Applying - In Memory
• Applying - On Disk

The incoming changes are grouped into two categories:

• Accumulating
• Applying

Each category is further broken down into two locations:

• In Memory
• On Disk

Incoming Changes is the total number of changes and transactions that are being read from the source database endpoint. In the picture below, you can see that 12 changes are read from 5 transactions.

The two Accumulating bars are a breakdown of the Incoming Changes. Accumulating means the Replicate server is reading the changes from the source database endpoint.

In the picture below, you will see the values set in the task settings that determine if the incoming changes will go to the In Memory bar or the On Disk bar. In this case, the task has been allocated to use a maximum of 1024MB of memory for the task, and the transaction duration is kept for a maximum of 60 seconds. Changes that exceed this memory limit or transaction duration are held in the On Disk bars.

Incoming changes that are held In Memory bar can be offloaded to the next step where Replicate will begin the apply process. Incoming changes that are held On Disk bar will wait for available memory before they are placed into the In Memory bar. 

As a reminder, the total number of changes/transactions in the two Accumulating bars will be equal to the Incoming Changes bar. The two Accumulating bars are just a breakdown of where the changes are currently stored. They are either in memory or on disk waiting for memory to start the processing.

The next step of the process is the two Applying bars. Applying means the Replicate server is gathering the changes on the Replicate server while it waits until the task settings have been reached and the batch of changes are applied to the target database endpoint.

Please refer to the Change Processing Tuning values you have set in the task settings to determine the wait time for the Replicate server before the batch of changes is applied to the target database. 

In the picture below, you will see that the task has been set to move a batch of changes to the apply process when a duration of 1-30 seconds has elapsed or the batch of changes has reached 500MB in size. Note that these are just the default values and are by no means the exact value you should set for the task. 

The In Memory and On Disk bars work the same way as the previous set of In Memory and On Disk bars. Up to 1024MB of memory can be used for the task before changes are stored on disk. As memory is freed up from changes being applied to the target, the changes will be moved from the On Disk bar to the In Memory bar which is then applied to the target in the next batch of changes.

Questions & Answers

When are changes captured from the source?

Changes are captured when the task is in a running state. The task will have a green circle with a white triangle icon. The task can also be in a stopped state with a grey circle and white square icon or an error state with a red circle and white X icon. The Stopped and Error state tasks will not capture changes anymore.

What happens to the changes when I stop the task in midst of the replication process?

Replicate will immediately stop capturing any more changes from the source database. Replicate will continue to apply changes to the target database for up to 30 minutes. After 30 minutes have passed, any remaining changes will be left in the Applying - On Disk bar. This means that changes are stored on the Replicate server disk. The remaining changes will be applied the next time the task is resumed.

What happens to the changes stored on the Replicate server disk if I choose to use the advanced run options to start the task from a timestamp instead of resuming the task?

If the "Start from a timestamp" option is used instead of the "Resume" option, all changes that are still stored on the Replicate server disk will be deleted. Replicate will start reading changes again from the chosen timestamp.

Will Replicate know which changes have not been applied to the target database or is there a chance Replicate will read the same changes twice?

Replicate uses an internal variable called stream position to keep track of changes that have been read. The stream position consists of the record position within the source database which lets Replicate know the exact location of the last change read. The stream position also lets Replicate know the exact record position that has been applied to the target. This allows Replicate to stop and resume without duplicating any source reads or target apply processes.

If I use the same source endpoint in another task, will the incoming changes count CDC from the other task too?

No, the monitoring tool UI that you see for the task only contains information pertaining to the single task you are viewing. Each task has its own separate set of information regarding the changes for the task.

Environment

• Qlik Replicate 

The information in this article is provided as-is and to be used at own discretion. Depending on tool(s) used, customization(s), and/or other factors ongoing support on the solution below may not be provided by Qlik Support.

This is a companion article to Qlik Sense Enterprise for Windows - New Security Patches Available Now  and only applies to the patches affected by QB-30633.

If you have experienced issues with visualizations and extensions not loading after upgrading to the security patches released on the 4th of December 2024, upgrade to the new patches which address QB-30633:

• May 2024 Patch 11
• February 2024 Patch 15
• November 2023 Patch 17
• August 2023 Patch 17
• May 2023 Patch 19
• February 2023 Patch 16

If visualizations and extensions do not behave as expected after the upgrade, follow the outlined steps below:

If the extensions are not available in the Qlik Sense Management Console, on the file share, as well as in the assets panel

Solution:

Restart the Qlik Sense Services services, then verify again

If the extensions are available in the Qlik Sense Management Console, on the file share, as well as in the assets panel, but not visible in any of your Qlik Sense apps

Solution:

Clear the browser cache

If sheets are not rendering and the error 'an error occured' is thrown

Solution:

Clear the browser cache

If issues continue regardless, contact support so that we can assist you effectively.

Environment

• Qlik Sense Enterprise on Windows May 2024 Patch 11
• Qlik Sense Enterprise on Windows February 2024 Patch 15
• Qlik Sense Enterprise on Windows November 2023 Patch 17
• Qlik Sense Enterprise on Windows August 2023 Patch 17
• Qlik Sense Enterprise on Windows May 2023 Patch 19
• Qlik Sense Enterprise on Windows February 2023 Patch 16

This document explains how to manually setup custom connectors within Qlik Sense with Standard mode without legacy mode. More info under Legacy vs. Standard Mode in Qlik Sense: Explainer and Security Considerations 

Pre-requisites:

• QlikView Desktop Client where SAP Connectors has been installed
• Qlik Sense Server installed and working perfectly
• Granted permission to copy files

After installing Qlik SAP Connectors on a Qlik Sense Enterprise Server, some connectors like Qlik SAP Report Connector, Qlik SAP Query Connector..., are not automatically installed. Thus, when migrating from a "QlikView environment" to "Qlik Sense Enterprise", those specific connectors need to be manually set in Qlik Sense.

Step 1:

1. Login to Qlik Sense Hub
2. Create a new app
   
   
   
   
3. Name the Application, then click Create button
   
   
   
   
4. Click Open app button
   
   
   
   
5. Click on Data load editor
   
   
6. Click on Create new connection button 
   
   
   
   
7. Click on OLE DB
   
   
   
   
8. Select OLE DB Provider for Microsoft Directory Service
   
   
   
   
9. Named the Data source as for example Dummy DataSource Connection. Choose the default selection, Specific user name and password
   
   
   
   
10. Put a name for the Connection, for instance, SAP Connection
    
    
    
    
11. Click Test connection button. Test must be successful similar to screenshot below
    
    
    
    

After successful connection, click Create button

Step 2: 

1. Open QlikView Desktop Client
2. Generate a connection string for this example QvSAPQueryConnector
3. CUSTOM CONNECT TO "Provider=QvSAPQueryConnector.dll;ASHOST=10.76.162.44;SYSNR=00;CLIENT=800;XUserId=xxxxxxxxxxxxx;XPassword=xxxxxxxxxxx;";

Step 3: 

1. Login to Qlik Sense Management Console (QMC)
2. Click on Data connections
   
   
   
   
3. Select SAP Connection, press Edit button found at the bottom left of the window
   
   
   
   
4. Remove the entry on the Connection string
   
   
   
   
5. Edit the generated connection string from QlikView Desktop
6. Remove the XUserID and XPassword parameter on the connection string

It should look like:

Step 4: 

• In the QMC, now paste the revised connection string to the Connection string field

• Provide the SAP user credential (UserID / Password), then click Apply button
  
  

Step 4:

• Login or go back to the document created earlier in the Qlik Sense Hub
• Click on Insert Script from Data connections (left corner of the Editor Script Window)
  
  
  
  
• Then, finally copy and paste the Load script generated in QlikView  Desktop to the Qlik Sense Editor window. Note that, in this scenario, you need to copy scripts from QlikView and not generate scripts using Qlik sense editor/select data.
  
  
  
  
• Save the application and press the Load data button
  
  

Optional Steps:

With the latest versions of QlikView and Qlik SAP Connector,  you don't  need to perform this step.

• Login to the machine which has QlikView Desktop Client
• Copy all the folders (by default) under C:\Program Files\Common Files\QlikTech\Custom Data\

• Login to the Qlik Sense server
• Copy over the folders earlier (by default) to C:\Program Files\Common Files\Qlik\Custom Data

• Restart all Qlik Sense services

You can access Qlik Sense demo apps from https://demos.qlik.com/.

If you are looking for real-life business examples, our Qlik Gallery hosts a platform meant for sharing apps, outcomes, and ideas. It is not restricted to Qlik-provided demos and mostly consists of customer examples.

If you are looking to download demo apps:

1. Go to https://demos.qlik.com/ 
2. Select Downloadable from the available Tags
   
3. Open the App
4. A Download button will be available (if logged in)
   
   

After a Qlik Sense upgrade, the Qlik Sense Repository service fails to start. The following error is logged in the Windows Application Event log:

Description: The application requested process termination through System.Environment.FailFast(string message).
Message: Unhandled exception: Could not load file or assembly 'Qlik.Sense.Logging, Version=37.3.0.0, Culture=neutral, PublicKeyToken=1a848309662c81e5' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
System.IO.FileLoadException: Could not load file or assembly 'Qlik.Sense.Logging, Version=37.3.0.0, Culture=neutral, PublicKeyToken=1a848309662c81e5' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
File name: 'Qlik.Sense.Logging, Version=37.3.0.0, Culture=neutral, PublicKeyToken=1a848309662c81e5'
at Repository.QRSService.Start(Action`1 terminate)
at

Resolution

The initial recommendation is to repair the installation. See How to repair an installation.

If the repair does not resolve the problem:

1. Copy the Qlik.Sense.Logging.dll from another environment of the identical Qlik Sense version, including patch version. The file can be located in: [Installation Qlik Sense folder]\Qlik\Sense\Repository\
2. Place the copied file in the same folder on the failing server
3. Restart the Qlik Sense Repository service.

Cause

The Qlik.Sense.Logging.dll present in [Installation Qlik Sense folder]\Repository\ was missing required inforamtion. 

Environment

Qlik Sense Enterprise on Windows 

Affected versions:

• Qlik Sense Enterprise on Windows all versions, including November 2024

CVE-2024-7348 is a Time-of-Check Time-of-Use (TOCTOU) race condition vulnerability in pg_dump, a utility used for backing up PostgreSQL databases. This vulnerability allows an attacker to replace a relation type (such as a table or sequence) with a view or foreign table right when pg_dump is running. Because pg_dump often runs with superuser privileges, this attack could execute arbitrary SQL code, leading to unauthorized actions or data corruption. Source: https://www.postgresql.org/support/security/CVE-2024-7348/

How does this vulnerability impact Qlik Sense Enterprise on Windows?

Qlik can confirm that pg_dump is not actively used in the Qlik Sense Enterprise on Windows code.

How to mitigate the vulnerability 

Both steps are required to fully mitigate the issue.

• Upgrade PostgreSQL. This requires a standalone instance of PostgreSQL. See Upgrading and unbundling the Qlik Sense Repository Database using the Qlik PostgreSQL Installer on how to unbundle PostgreSQL if necessary.

  Always verify compatibility between your Qlik Sense version and PostgreSQL before planning an upgrade.

• Delete the pg_dump.exe located in the default Qlik Sense Enterprise on Windows Postgresql install folder: C:\Program Files\Qlik\Sense\Repository\Postgresql\14\
  
  The updated pg_dump.exe in, for example, C:\program files\postgresql\14 does not need to be removed. 

Does the Qlik PostgreSQL Installer (QPI) use the pgdump.exe?

QPI does not utilise pgdump.exe.

Upgrading Qlik Sense Enterprise on Windows fails with a generic error message:

An error has occurred
For detailed information see the log file

The Qlik Sense Repository Database service is no longer installed after the failed installation.

This means that the database password validation did not complete and a partial rollback has occurred. This has removed the database software, but it has not removed the database itself.

The database password is normally password is checked before the upgrade process begins, but under some circumstances, such as the database service being stopped, this check does not occur. Once the new version of the database software is installed it will attempt to open the database as part of the install verification process and will fail. This then prompts a rollback of the database software and removing the new version.

Resolution:

Reinstall Qlik Sense. (Uninstall, Reinstall)

During the uninstall process, do not remove the certificates or database. 

Once Qlik Sense is uninstalled, you can proceed to install the desired upgraded version of Qlik Sense, the installer will detect the existing database and request the password, if it is from an earlier version the installer will make the necessary changes as part of the installation process.

Environment:

Qlik Sense Enterprise on Windows 

When using context or global map to pass values to subjobs or other child jobs components, the  record passing is showing data inconsistency issue under multi-thread scenarios.

Considering the various factors, you need to choose a Thread Safe and Mutable Object.  In this case, try to create a ConcurrentHashMap (which is an implementation of Map) as a single Object that can be passed to all child Jobs or other components and it can also accept parameters we want to add. In fact, this provides similar functions to the globalMap, but the globalMap has no thread-safe guarantee.

globalMap.put("sharedMap", new java.util.concurrent.ConcurrentHashMap());

((java.util.Map) globalMap.get("sharedMap")).put("firstname","xxx");

System.out.println("firstname=" + ((String) ((java.util.Map) globalMap.get("sharedMap")).get("firstName")));

Related Content

• Passing a value from a parent Job to a child Job
• Returning a value from a child Job to the parent Job

Environment

• Talend Studio 
• Talend Data Integration 

When you connect to the Qlik Sense Enterprise on Windows hub or the Qlik Cloud Analytics hub with Firefox and open an app, the browser automatically shows a window suggesting a translation from Greek to the OS/Qlik language, as shown in this image. 

Resolution

The suggestion provided by Firefox can be ignored by clicking Cancel.

This has no impact on the functioning of Qlik.

Cause

The message is shown by Firefox, which can incorrectly detect Greek language contents in Qlik apps. The problem has been reported to Firefox developers.

Internal Investigation ID(s)

QB-29728

Environment

• Firefox
• Qlik Cloud Analytics
• Qlik Sense Enterprise on Windows

This article explains how to simply set up JWT authentication using Qlik Sense default certificates and test it.

Steps:

1. Create a new virtual proxy
   
   This section only explains settings specific to JWT and supposes that you have set up Name, Prefix, Session cookie header name, load balancing nodes and linked a Proxy to the new virtual proxy.
   These are the minimum settings required for a virtual proxy to work correctly. Please refer to the Qlik Sense Online Help "Creating a virtual proxy" as well as Qlik Sense For Administrators - JWT authentication for details.
   
   
   1. In the authentication section, select JWT for authentication
   2. Paste the certificate used to sign/decrypt JWT in "JWT certificate
      
      In this article, we will use the default Qlik Sense certificate located:
      
      Important note: We use Qlik Default certificates for simplicity in this article, in a Production scenario, the customer should create the key material (generate a private/public key pair for JWT signing and verification) that is managed by customer internal security policies.
      
      
   3. Go to C:\ProgramData\Qlik\Sense\Repository\Exported Certificates\.Local Certificates
   4. Open server.pem in a text editor
   5. Copy the content including -----BEGIN CERTIFICATE----- and -----END CERTIFICATE----- then paste it in the JWT Certificate field in the QMC > Virtual Proxy configuration.
   6. Choose a value for JWT attribute for user ID and JWT attribute for user directory. That can be anything (except reserved key words for JWT claims) but it must match the values used in the JWT payload section when the JWT is generated.
   7. Apply changes and close.
      
       

      

      
      
      
2. Generate the JWT
   
   In this article, we will directly use the JWT debugger available on https://jwt.io to generate the token. In a real use case, the JWT library corresponding to the programming language used should be used to generate the JWT.
   
   
   1. Open https://jwt.io and go to the debugger
   2. Select RS256 for the algorithm
   3. In the PAYLOAD box, set up your attributes previously set via QMA in Qlik Sense's virtual proxy configuration for the user ID and the user directory. (Note: Swap 'DOMAIN' with the user's directory) 
   4. In the Verify signature, paste server.pem (Certificate) and server_key.pem (Private key) content in the appropriate fields. Those files are both located in C:\ProgramData\Qlik\Sense\Repository\Exported Certificates\.Local Certificates
   5. You can now copy the JWT generated in the left field and test it. This should look like:
      
      
      
      
3. Test the generated JWT
   
   In order to log in with JWT, you need to inject the JWT as a header. There are various extensions available depending on your browser in order to test this.
   
   In this example, we are using Chrome with Modheader.
   
   
   1. Install ModHeader from the Chrome Web Store
   2. Try to go to the hub:
      
                https://{your server name}/{prefix for JWT virtual proxy}/hub/
      
      
   3. You should get a 401 error: "Could not authenticate the request: Expected an authentication header." as no header has been provided.
   4. Go to ModHeader and set the following header:
      
             Header name: Authorization
             Header value: Bearer yourjwt
      
      
      
      
   5.    Reload the page, you should now be logged into the hub correctly.

In order to integrate your solution with Qlik Sense using JWT authentication, you will need to pass in your code the JWT token in the authorization token for the first request to Qlik Sense so that a session is created.

Related Content:

• Using Qlik Sense on Windows Repository API (QRS) with qlik-cli 
• Examples of integration for different purposes are available on the internet:
  https://github.com/qlik-oss/enigma.js/tree/master/examples/authentication/sense-using-jwt

The following error is showing when executing a job on Talend Administration Center from MetaServletCaller.bat

{"errorStatus":"CONNECTION_TO_SERVER_FAILED","execBasicStatus":"UNKNOWN","execDetailedStatus":"UNKNOWN","execDetailedStatusLabel":"Unknown, JobServer might be unreachable","execRequestId":"****","executionTime":{"millis":4817930,"seconds":4817},"returnCode":0,"status":"UNKNOWN"}

Resolution

• As mentioned in the error message, please check if you can connect to the JobServer from your Talend Administration Center and check if there is any network issue happening at the time.
• Check the status what you have set for “On unavailable JobServer” option and change it to a status in which the job will run automatically when the server is unavailable during a predetermined period of time.
• The issue may be related to the patch you have installed. Please check on patch notes if there are any problems that have been fixed, and consider upgrading your Talend Administration Center to a newer version.
  

When considering upgrades, please upgrade your Job Server to the same version as the Talend Administration Center's, since Talend Administration Center and Job Server will interact with each other in work and installing an old version from one of these two applications may cause issues

Cause

• Network issues that cause problem to the connection between Talend Administration Center and Job Server.
• Old patch for Talend Administration Center.
   

Related Content

• For more information about "On unavailable Job Server" option, please refer to documentation: Adding-execution-task-to-pre-generated-job
• Talend Patch Notes: TPS-5554 (cumulative patch) 

Internal Investigation ID(s)

Internal Jira case TAC-19340

Environment

• Talend Data Integration 
• Talend Administration Center 

This article explains how the Qlik Sense app button component can be used to send custom parameters directly to the automation without requiring a temporary bookmark. This can be useful when creating a writeback solution on a big app as creating and applying bookmarks could take a bit longer for big apps which adds delays to the solution. More information on the native writeback solution can be found here: How to build a native write back solution.

Contents

• Button configuration
• Automation configuration
• Limitations
• Related Content

Button configuration

1. In this example, we'll use an app that contains car sales opportunities from Salesforce. The straight table, the variable input dropdown, and the native button are important for this example.
   

   
   

2. Start editing the sheet and go to the button configuration section, set the action to "Execute automation" and specify the automation that should run when the button is clicked. If your automation isn't shown, you can copy and paste the automation id.
   
   
   
   
3. Additional input parameters can now be added through the "Add parameter" button.
   
   

   

4. Parameters require a key and a value, it's possible to use Qlik Expressions as input for the value. These expressions are evaluated when the button is clicked.
   
   
   
   
5. After adding the parameters, use the "Copy input block" button to copy the all parameters that will be sent to the automation to your clipboard.
   
   
   
   

Automation configuration

1. Configure the automation's run mode to "Triggered"
   

   
   

2. Right-click in the automation canvas and select "Paste Block(s)" to add the Inputs block to the automation. Connect the new Inputs block to the Start block. Apart from the default parameters, you'll also find your custom parameters as Inputs in the block.
   
   

   

3. You can now reference the custom parameters from the button in other blocks in the automation by selecting the output from the Inputs block.
   
   
   
   
   

    

Limitations

1. The maximum payload size when triggering an automation is 32kb. When using the GetCurrentSelections expression, every field value will be sent to the automation. This could exceed the 32kb limit. Use the temporary bookmark solution as described in this article instead: Native writeback solutions.
2. Other automation limitations are available here: Platform limitations.
3. More info on configuring the native button can be found here: Creating buttons.

Related Content

• Creating buttons
• Native writeback solutions

Environment

• Qlik Cloud
• Qlik Application Automation

The information in this article is provided as-is and will be used at your discretion. Depending on the tool(s) used, customization(s), and/or other factors, ongoing support on the solution below may not be provided by Qlik Support.

When running the Job that includes a tFTPConnection component, the following error occurs:

Error: Exception in component tFTPConnection_1 com.jcraft.jsch.JSchException: Algorithm negotiation fail.

Cause

The dependent JSCH jar 0.2.18 is missing.

Resolution

To resolve this issue, kindly follow these steps:

1. Download the latest JSCH jar from https://mvnrepository.com/artifact/com.github.mwiede/jsch/0.2.18
2. After downloading, add the tLibraryLoad component to load this new version of the JSCH jar before tFTPConnection.
3. Run the Job again.

Environment

• Talend Data Integration 7.3.1

QlikView and Qlik Sense provides the capability to extend presentation capability by creating extension objects.

It is important to note that Qlik only supports the API calls made to the QlikView or Qlik Sense server. Qlik does not directly support 3rd party extension implementations. For issues with 3rd party provided extensions, the primary support channel is the 3rd party extension provider. 

For Qlik to support API-related issues, the script or code must be minimised to show the issue is related to the specific API and not the custom code using the API. This initial troubleshooting and simplification of the script must be done by the extension developer. 

Extensions provided by Qlik employees on Qlik Community are not supported by Qlik. These extension solutions are supported within these communities.  

Extensions developed by Qlik’s partners, including certified Extensions, are supported by the Qlik partner.

A customer should also consider this:

• Extensions are generally not supported by Qlik. Qlik Support can only provide recommendations and general configuration advice for extensions. 
• Extensions should be tested one by one when installing them. This is meant to verify them working and not to break standard functionality. 
• A QlikView or Qlik Sense upgrade might cause conflict with non-compatible extensions. This can lead to Qlik product malfunction. It is recommended to always validate upgrade and extension compatibility in the test environment prior to upgrading the production environment. Any compatibility issues or questions should be reported to the extension provider. 

Talend Remote Engine Heart Beat is failing hourly with 'GOAWAY received' error and the karaf.log shows below:

2024-08-01T10:16:58,638 | ERROR | pool-28-thread-1 | JAXRSUtils | 469 - org.apache.cxf.cxf-rt-frontend-jaxrs - 3.6.2 | | Problem with writing the data, class org.talend.ipaas.rt.engine.model.HeartbeatInfo, ContentType: application/json
2024-08-01T10:16:58,638 | WARN | pool-28-thread-1 | PhaseInterceptorChain | 467 - org.apache.cxf.cxf-core - 3.6.2 | | Interceptor for {http://pairing.rt.ipaas.talend.org/}PairingService has thrown exception, unwinding now
org.apache.cxf.interceptor.Fault: Problem with writing the data, class org.talend.ipaas.rt.engine.model.HeartbeatInfo, ContentType: application/json
Caused by: java.io.IOException: /<IP>:<port>: GOAWAY received

Resolution

1. Stop Remote Engine.
2. Edit the <RemoteEngineInstallation>/etc/org.talend.ipaas.rt.pairing.agent.cfg file and replace the line 

   heartbeat.interval=60 (by default) 

   with 

   heartbeat.interval=65

3. Restart Remote Engine.

Do not set heart interval to more than 180 seconds(3 mins). If Talend Cloud does not receive a heartbeat from the Remote Engine for more than 3 mins, it will show the Remote Engine's status as unavailable.

Cause

It seems that the connection to the server pair.xx.cloud.talend.com is kept during 60 seconds and Heartbeat.interval > 60 seconds allows to close this connection and prevent "GOAWAY received" error message.

Related Content

For more information about heartbeat concept and heartbeat interval, please refer to documentation:

Monitoring-remote-engine-health

Internal Investigation ID(s)

 Internal Jira Case ID: TMC-4122

Environment

Talend Cloud