Upgrading and unbundling the Qlik Sense Repository Database using the Qlik PostgreSQL Installer

[Mario_Petre]

In this article, we walk you through the requirements and process of how to upgrade and unbundle an existing Qlik Sense Repository Database (see supported scenarios) as well as how to install a brand new Repository based on PostgreSQL. We will use the Qlik PostgreSQL Installer (QPI).

For a manual method, see How to manually upgrade the bundled Qlik Sense PostgreSQL version to 12.5 version.

Using the Qlik Postgres Installer not only upgrades PostgreSQL; it also unbundles PostgreSQL from your Qlik Sense Enterprise on Windows install. This allows for direct control of your PostgreSQL instance and facilitates maintenance without a dependency on Qlik Sense. Further Database upgrades can then be performed independently and in accordance with your corporate security policy when needed, as long as you remain within the supported PostgreSQL versions. See How To Upgrade Standalone PostgreSQL.

Index

• Supported Scenarios
• Upgrades
• New installs
• Requirements
• Installing anew Qlik Sense Repository Database using PostgreSQL
• Qlik PostgreSQL Installer - Download Link
• Upgrading an existing Qlik Sense Repository Database
• The Upgrade
• Compatibility with PostgreSQL installers
• Troubleshooting and FAQ
• Video Walkthrough
• Related Content

Supported Scenarios

Upgrades

The following versions have been tested and verified to work with QPI 1.3.0 (QPI May 2023): 

Qlik Sense February 2022 to Qlik Sense November 2023.

If you are on any version prior to these, upgrade to at least February 2022 before you begin.  

Note that November 2022 and later do not support 9.6, and a warning will be displayed during the Qlik Sense upgrade. 

New installs

The Qlik PostgreSQL Installer supports installing a new Qlik Sense Repository Database cluster on a new server, setting up a new environment or migrating an existing Database to a separate host.

Requirements

• Review the QPI Release Notes before you continue
• Using the Qlik PostgreSQL Installer on a patched Qlik Sense version can lead to unexpected results. If you have a patch installed, either:
  • Uninstall all patches before using QPI or
  • Upgrade to an IR release of Qlik Sense which supports QPI
• This installer will not detect a standalone PostgreSQL database installed by any other means, other than through an official Qlik installer package.
• The user who runs the installer must be an administrator.
• The backup destination must have sufficient free disk space to dump the existing database  
• The backup destination must not be a network path or virtual storage folder. It is recommended the backup is stored on the main drive. 
• The installer itself does not provide an automatic rollback feature.
• There will be downtime during this operation, please plan accordingly 

Installing a new Qlik Sense Repository Database using PostgreSQL

1. Run the Qlik PostgreSQL Installer as an administrator 
   
   
2. Click on Install 
   
   
3. Accept the Qlik Customer Agreement 
   
   
4. Set your Local database settings and click Next. You will use these details to connect other nodes to the same cluster. 
   
   
5. Set your Database superuser password and click Next
   
   
6. Set the database installation folder, standard: C:\Program Files\PostgreSQL\14
   
   
7. Set the database data folder, standard: C:\Program Files\PostgreSQL\14\data
   
   
8. Review your settings and click Install, then click Finish
   
   
9. Start installing Qlik Sense Enterprise Client Managed. Choose Join Cluster option.
   
   The Qlik PostgreSQL Installer has already seeded the databases for you and has created the users and permissions. No further configuration is needed.
   
   
10. The tool will display information on the actions being performed. Once installation is finished, you can close the installer.  
    
    If you are migrating your existing databases to a new host, please remember to reconfigure your nodes to connect to the correct host. How to configure Qlik Sense to use a dedicated PostgreSQL database 
    

Qlik PostgreSQL Installer - Download Link

Download the installer here.

Qlik PostgreSQL installer version 1.3.0 Release Notes

 

Upgrading an existing Qlik Sense Repository Database

The following versions have been tested and verified to work with QPI 1.3.0 (May 2023): 

February 2022 to November 2023.

If you are on any version prior to these, upgrade to at least February 2022 before you begin.  

Note that November 2022 and later do not support 9.6, and a warning will be displayed during the Qlik Sense upgrade. 

The Upgrade

1. Stop all services on rim nodes 
   
   
2. On your Central Node, stop all services except the Qlik Sense Repository Database 
   
   
   
   
3. Run the Qlik PostgreSQL Installer. An existing Database will be detected. 
   
   
   
   
4. Highlight the database and click Upgrade 
   
5. Read and confirm the (a) Installer Instructions as well as the Qlik Customer Agreement, then click (b) Next.
   
   
   
   
6. Provide your existing Database superuser password and click Next.
   
   
7. Define your Database backup path and click Next.
   
   
   
   
8. Define your Install Location (default is prefilled) and click Next.
   
   
   
   
9. Define your database data path (default is prefilled) and click Next.
   
   
    
10. Review all properties and click Upgrade.
    
    The review screen lists the settings which will be migrated. No manual changes are required post-upgrade.
    
    
    
    
11. The upgrade is completed. Click Close.
     
12. Open the Windows Services Console and locate the Qlik Sense Enterprise on Windows services.
    
    You will find that the Qlik Sense Repository Database service has been set to manual. Do not change the startup method. 
    
    
    
    You will also find a new postgresql-x64-12 service. Do not rename this service. 
    
    
    
    
13. Start all services except the Qlik Sense Repository Database service. 
    
    
14. Start all services on your rim nodes.
    
    
15. Validate that all services and nodes are operating as expected. The original database folder in C:\ProgramData\Qlik\Sense\Repository\PostgreSQL\X.X_deprecated
    

16. Uninstall the old Qlik Sense Repository Database service.

    This step is required. Failing to remove the old service will lead the upgrade or patching issues.

    1. Open a Windows File Explorer and browse to C:\ProgramData\Package Cache
       
       
    2. From there, search for the appropriate msi file.
       
       
       
       If you were running 9.6 before the upgrade, search PostgreSQL.msi
       
       If you were running 12.5 before the upgrade, search PostgreSQL125.msi
       
       
    3. The msi will be revealed.
       
       
    4. Right-click the msi file and select uninstall from the menu.
       
       
       
       
    
    
17. Re-install the PostgreSQL binaries. This step is optional if Qlik Sense is immediately upgraded following the use of QPI. The Sense upgrade will install the correct binaries automatically. 
    

    Failing to reinstall the binaries will lead to errors when executing any number of service configuration scripts. 

    1. Open a Windows Command prompt as administrator
       
       
    2. Navigate to the location of the previously located .msi file
       
       Example: 
       
       cd C:\ProgramData\Package Cache\{GUID}
       
       
    3. Run the following command, replacing PostgreSQL125.msi with the correct .msi depending on your previous version used.
       
       Qlik Sense August 2023 and later: PostgreSQL14.msi
       Qlik Sense February 2022 to May 2023: PostgreSQL125.msi
       
       msiexec.exe /qb /i "PostgreSQL125.msi" SKIPINSTALLDBSERVICE="1" INSTALLDIR="C:\Program Files\Qlik\Sense"
       
       This will re-install the binaries without installing a database.

If the upgrade was unsuccessful and you are missing data in the Qlik Management Console or elsewhere, contact Qlik Support.  

Compatibility with PostgreSQL installers

This version of the upgrade tool is fully compatible with the official PostgreSQL installers from https://www.enterprisedb.com/downloads/postgres-postgresql-downloads.

If you have previously used the Qlik Postgres Installer (version 1.2.1 or earlier), you can simply install the latest PostgreSQL version (within your major release).

Example: You have used the old QPI to upgrade to 12.5. You can now easily upgrade to a later version in the same major release, such as 12.15.

Troubleshooting and FAQ

1. If the installation crashes, the server reboots unexpectedly during this process, or there is a power outage, the new database may not be in a serviceable state. Installation/upgrade logs are available in the location of your temporary files, for example:
   
   C:\Users\Username\AppData\Local\Temp\2
   
   A backup of the original database contents is available in your chosen location, or by default in:
   
   C:\ProgramData\Qlik\Sense\Repository\PostgreSQL\backup\X.X
   
   The original database data folder has been renamed to:
   
   C:\ProgramData\Qlik\Sense\Repository\PostgreSQL\X.X_deprecated
   
   
2. Upgrading Qlik Sense after upgrading PostgreSQL with the QPI tool fails with:
   
   This version of Qlik Sense requires a 'SenseServices' database for multi cloud capabilities. Ensure that you have created a 'SenseService' database in your cluster before upgrading. For more information see Installing and configuring PostgreSQL.
   
   See Qlik Sense Upgrade fails with: This version of Qlik Sense requires a _ database for _.
   
   To resolve this, start the postgresql-x64-XX service.

Video Walkthrough

The information in this article is provided as-is and to be used at own discretion. Depending on tool(s) used, customization(s), and/or other factors ongoing support on the solution below may not be provided by Qlik Support. The video in this article was recorded in a earlier version of QPI, some screens might differ a little bit.

Related Content

Qlik PostgreSQL installer version 1.3.0 Release Notes  
Techspert Talks - Upgrading Qlik Sense Repository Service
Backup and Restore Qlik Sense Enterprise documentation
Migrating Like a Boss
Optimizing Performance for Qlik Sense Enterprise
Qlik Sense Enterprise on Windows: How To Upgrade Standalone PostgreSQL
How-to reset forgotten PostgreSQL password in Qlik Sense
How to configure Qlik Sense to use a dedicated PostgreSQL database
Troubleshooting Qlik Sense Upgrades 

[dwighttrumbower]

This tool did work to upgrade PostgreSQL.  I did copy the old bin directory to the c: drive from d:. It appears it uses the c: location to do the backup.

System was running well with Nov 2022.

Did the upgrade to Feb 2023 and that did not work. Now I have a sever that does not have any of the services listed in service manager and I'm waiting for support to continue.  They have my info and logs. 

Server is on a VM and will probably need to restore the snapshot.  Too bad, first real issue with an upgrade since 3.x.

[Sebastian_Linser]

@fabdulazeez this would normally only have happened when you use the very first version of QPI released, but not with 1.2.1. To get a better understanding please contact support and we will fix that with you.

[fabdulazeez]

Thanks Sebastian.

Yes we had used QPI as part of early access program. I have contacted support and they have provided steps to change the folder path for the database.  We have upgraded successfully.

[cshimodo]

Our customer wants to install PostgreSQL on drive D after migration. In this case, should we specify the path to the D drive in step 8? Is this supported?

[Sebastian_Linser]

 @cshimodo Hello, yes that already works, you can install the new PostgreSQL instance with data and bin folder on any drive.

[lachlanwwells]

Would be good if the newly installed postgreSQL 12 database that is installed by the QPI tool has a 'Qlik' friendly name - instead of the default 'postgresql-x64-12' - can it not just replace the 'Qlik Repository Database' service and installation path?

[cshimodo]

Thanks for the reply Sebastian. I have an additional question.
If PostgreSQL 12.5 has already been installed on the C drive, is it possible to move it to the D drive?

[Serkan]

Hello @Sebastian_Linser 

is there anything wrong when first crate a backup (backup respository --> DB dump), then uninstall QS and install the latest version. After Installation create an restore of the DB?

So you have a standard installation without deviation and without another service (PostgreSQL).

Thanks in advance for your reply.

[Sebastian_Linser]

@Serkan that is still a valid way to go. the only downside is that you have no control over the Postgres version, meaning you can't quickly upgrade to 12.11 if you needed some security issue fixed. 

 @cshimodo in theory that should be possible see https://radumas.info/blog/tutorial/2016/08/08/Migrating-PostgreSQL-Data-Directory-Windows.html and https://wiki.postgresql.org/wiki/Change_the_default_PGDATA_directory_on_Windows

even if both links show an older version, the system is still the same. But you have to run a few tests like a uninstall etc to see that it doesn't have side effects. 

regards Sebastian

[steeefan]

@Sebastian_Linser, I'm referring to my post about stream IDs having changed after upgrading. I never opened a support ticket about this because I had no proof. 

I finally upgraded two more servers, Dev and Prod, to March 2023. This time I recorded the stream IDs beforehand and compared with those after the upgrades. This time it did not happen, all stream IDs remain the same as they were before the upgrade.