article How to configure Qlik Sense to use a dedicated PostgreSQL database in Official Support Articles

How to configure Qlik Sense to use a dedicated PostgreSQL database

Sebastian_Linser — Thu, 08 Feb 2024 12:58:47 GMT

This article is provided as-is and to be used at own discretion. Depending on tool(s) used, customization(s), and/or other factors ongoing support on the solution below may not be provided by Qlik Support.

The Qlik Sense Repository Database (QSR) can be moved to a dedicated standalone PostgreSQL instance not hosted on the same machine as other Qlik Sense Services. It is also possible to move an already existing QSR from a local Sense install, to a dedicated PostgresSQL database.

Content:

Set up a postgresSQL database

Instructions for this might vary slightly. Please refer to the Qlik Online Help for details. We recommend watching this video.

Qlik Sense Enterprise server has a set of five databases. For more details visit Installing and configuring PostgreSQL on help.qlik.com.

1. The Qlik Sense repository database (QSR):

The QSR is the primary database in your Qlik Sense deployment.

2. The Qlik Sense services database (SenseServices):

The SenseServices database contains schemas for each of the Qlik Sense services and allows growth independently of the Qlik Sense Repository Database, while still sharing the same PostgreSQL instance and login role.

3. The Qlik Sense message queue database (QSMQ):

The QSMQ database provides a light-weight method of passing messages internally between services in Qlik Sense Enterprise. The NOTIFY and LISTEN functionality in PostgreSQL allows services to be notified about new messages that have been written to the messaging table.

4. The Qlik Sense logging database (QLogs): No longer supported after May 2021

The QLogs database centralizes logging by collecting log messages from all Qlik Sense nodes in your deployment and stores them in a PostgreSQL database.

5. The Qlik Sense license service database (Licenses):

The Licenses database is used to sync the assignments between the Qlik license backend and the local installation.

Simplified checklist of steps:

Download a valid version of postgresSQL. For Sense June 2017 and up, 9.6.x is required, from November 2019 version 11.5.x as well. From May 2021 version 12.5 can be used. Review the system requirements of your version before proceeding.
Install based on instructions found on the Qlik Online Help for Qlik Sense; select the version you are running on then select from Menu , Deploy - Qlik Sense installation - Installing and configuring PostgreSQL.

See Installing and configuring PostgreSQL

Moving the Qlik Sense Share Folders as well?

Follow How to change the shared path in Qlik Sense

Backup

Depending on the version, you need to backup different databases, on the current Qlik Sense central node hosting the Qlik Sense Repository Database service, follow the backup instructions to obtain a database copy of each Qlik Sense Database Present .
Example for February 2021 you need to backup the databases reported in section "Installing and configuring PostgreSQL"; referring to the Help site these are :
QSR , QSMQ , SenseServices , Licenses , if configured QLogs

Restore

On the dedicated remote postgresSQL host, perform a restore of the backed up Databases of previous point.

Changing the Connection String to QSR and QSMQ Database

On the Qlik Sense central node, all services but the Qlik Sense Repository Service are shut down.
Open QlikSenseUtil. By default, this will be installed in: C:\Program Files\Qlik\Sense\Repository\Util\QlikSenseUtil\QlikSenseUtil.exe
Open the Connection String Editor and Read the current configuration, decrypting the current data.
Replace all mentions of the current database hostname. In our example, the new database resides on 172.16.16.103. Instead of an IP you can also use the FQDN of the db server.
You need to repeat this step for the sections <add name="QSMQ" and <add name="postgres"
Hit Save Value in config file encrypted, committing any changes.

Changing the Connection String to SenseServices Databases.

Execute steps in Moved Postgress database to new host, but some qliksense is still accessing the SenseServices old database; check if different version of article are present for different Qlik Sense version .

Changing the Connection String to Licenses Database ( April 2019 and up )

Starting from April 2019 new License Service implementation is using this database.

Open an elevated Powershell and change directory to C:\Program Files\Qlik\Sense\Licenses
Run the script .\Configure-Service.ps1 and provide:
- new Hostname or IP
- database port default is 4432
- database user qliksenserepository
- databasepassword is the qliksenserepository password used during configuration of the new database.
example:
cd 'C:\Program Files\Qlik\Sense\Licenses'
.\Configure-Service.ps1 172.16.16.102 4432 qliksenserepository Password123!

It will look like this:

Windows PowerShell
Copyright (C) 2014 Microsoft Corporation. All rights reserved.

PS C:\Windows\system32> cd \
PS C:\> cd 'C:\Program Files\Qlik\Sense\Licenses'
PS C:\Program Files\Qlik\Sense\Licenses> .\Configure-Service.ps1 172.16.16.102 4432 qliksenserepository Password123!
Licenses configuration started.
WARNING: Skipping the database initialization. No superuser or password specified.
Reading the settings file.
Saving the modified settings.
Licenses configuration successful.
PS C:\Program Files\Qlik\Sense\Licenses>

repeat this for:

C:\Program Files\Qlik\Sense\NotifierService\install\Configure-Service.ps1
C:\Program Files\Qlik\Sense\AppDistributionService\Configure-Service.ps1
C:\Program Files\Qlik\Sense\HybridDeploymentService\Configure-Service.ps1
C:\Program Files\Qlik\Sense\MobilityRegistrarService\install\Configure-Service.ps1
C:\Program Files\Qlik\Sense\PrecedentsService\install\Configure-Service.ps1
C:\Program Files\Qlik\Sense\NLAppSearch\install\Configure-Service.ps1

there might come new services or disappear with new releases, please check in the C:\Program Files\Qlik\Sense\ folder for "Configure-Service.ps1" to find additional one in your version of Qlik Sense.

Note: If you have installed Qlik Sense in a different directory the Configure-Service.ps1 would have to be updated as in this example:

File: Configure-Service.ps1 Folders: Various (9 microservices)
Before:
11 | [string] $senseInstallPath
"$env:ProgramFiles\Qlik Sense",
After:
11
[string] $senseInstallPath = "D:\Qlik",

Remove Database Service dependency and Postgres Server

After successfully verifying your new setup, remove the Qlik Sense Repository service dependencies and the old Postgres database. This is done to avoid issues with future Upgrade of Environment.

In an elevated command prompt, execute the following on prompt:

c:\sc config "QlikSenseRepositoryService" depend= ""
c:\sc config "QlikLoggingService" depend= ""

This will return:

[SC] ChangeServiceConfig SUCCESS
Restart the Qlik Sense Repository Service.

Check that Qlik Sense Repository Database doesn't start (confirms [SC] ChangeServiceConfig SUCCESS ).
Remove the (local) Qlik Sense Repository Database

Find the most recent PostgreSQL.msi in C:\ProgramData\Package Cache\ and remove it. This removes the service but no old local database data. This is needed to prevent issues on future upgrades of Environment.

You can find the precise folder with this command depending on the Postgresql Version you used installing the inital release.

9.6:

c:\cd ProgramData\Package Cache
c:\ProgramData\Package Cache\dir /s PostgreSQL.msi

The folder containing PostgreSQL.msi will be revealed. Right click on the PostgreSQL.msi file and select uninstall from the menu.

12.5:

c:\cd ProgramData\Package Cache
c:\ProgramData\Package Cache\dir /s PostgreSQL125.msi

The folder containing PostgreSQL.msi will be revealed. Right click on the PostgreSQL.msi file and select uninstall from the menu.

Finalize

Restart the Qlik Sense Repository Service and verify that Qlik Sense Repository Database Service doesn't exist when followed the previous steps.
Start all remaining Sense Services
Progress on the startup sequence (including the connection to the new database) can be tracked in the _System_Repository log files located in C:\ProgramData\Qlik\Sense\Log\Repository\Trace
Verify functionality.
In Multi-node scenario you need to repeat steps "Changing the Connection Strings to Databases" on all Rim nodes (QlikSenseUtil for Repository.exe.config and Configure-Service.ps1 for the micro services), and if required configure "Update the Qlik Logging Service Connection String" (note that is EOL with Qlik Sense Enterprise on Windows February 2022) .

Environment

Re: How to configure Qlik Sense to use a dedicated PostgreSQL database

aadil_madarveet — Thu, 28 Oct 2021 14:31:58 GMT

Hi,

I am running Qlik Sense Feb 2021 P8.

I am trying to migrate the PostgreSQL db to a separate standalone node to implement HA.

I have followed your instructions step by step.

All the services are running now, but somehow the proxy is not kicking in. Still shows this error in the system proxy log.

The proxy configurations are still pointing to localhost for QlikSenseDatabaseService.

24 20211028T142549.340+0000 ERROR server System.Proxy.Proxy.Communication.Rest.ProxyRestClient 9 59602e12-d054-4368-bea7-0f0f4062e0a8 DEVSERVER\senseadmin When contacting https://localhost:4242/ No connection could be made because the target machine actively refused it [::1]:4242↵↓Unable to connect to the remote server at System.Net.Sockets.Socket.InternalEndConnect(IAsyncResult asyncResult)↵↓ at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)↵↓ at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Exception& exception)↵↓ at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)↵↓ at Qlik.Sense.Common.Communication.REST.Client.WebClientEx.GetWebResponse(WebRequest request, IAsyncResult result)↵↓ at System.Net.WebClient.DownloadBitsResponseCallback(IAsyncResult result) 59602e12-d054-4368-bea7-0f0f4062e0a8

After following this step,

"Right click on the PostgreSQL.msi file and select uninstall from the menu."

The QlikSenseRespositoryDatabase service has got removed from the box. It does not appear on the server anymore.

Re: How to configure Qlik Sense to use a dedicated PostgreSQL database

Sebastian_Linser — Fri, 29 Oct 2021 12:45:50 GMT

Hello @aadil_madarveet

since the response in the logs is that you can't reach the repository service, it seems that this one is not up. So please check in the repository logs first why the DB is not connecting towards your new instance.

best regards

Sebastian

Re: How to configure Qlik Sense to use a dedicated PostgreSQL database

Lokeshb31 — Tue, 04 Oct 2022 06:24:57 GMT

Hi,

We are running Qliksense Feb 2021 Patch3 on Windows Server 2019. We want to move qliksense repository to new DB host for High Availability. AS per backup instructions stated in https://help.qlik.com/en-US/sense-admin/August2022/Subsystems/DeployAdministerQSE/Content/Sense_DeployAdminister/QSEoW/Deploy_QSEoW/Backing-up-a-site.htm I am taking a backup but shall we backup and restore all DBs like SenseServices, QSMQ, Licenses? We are not using centralized logging and QSMQ/SenseServices are asked to be backuped up in case of linked cloud environment(What does it mean? We have on-prem Qliksense installation)

Will it be ok if we only restore QSR to another DB, update only that configuration in Qliksenseutil.exe and it will work? Do I need to delete default Qliksense Repository database service in that case or I can keep it up?

Please help to address this confusion.

Re: How to configure Qlik Sense to use a dedicated PostgreSQL database

Sebastian_Linser — Tue, 04 Oct 2022 08:16:19 GMT

@Lokeshb31

The basic principle is that what you marked above

- if you have a could deployment, so connection to tennantid.(eu|us|ap).qlikcloud.com and you are pushing apps from Qlik Sense into this instance you need the SenseServices and QSMQ table,

- if you use a signed license key (SLK) then you need the Licenses database if you use LEF and Controlnumber (which is highly likley in Feb2021) then you dont need this database.

Since you migrate you have at least to create empty DB's, the script here works just fine.

https://help.qlik.com/en-US/sense-admin/August2022/Subsystems/DeployAdministerQSE/Content/Sense_DeployAdminister/QSEoW/Deploy_QSEoW/Installing-configuring-postgresql-on-Azure.htm

If you just move the only QSR the worst what can happen is that you reconfigure the cloud tennant one more time in the QMC, the rest will work just fine.

Beside the Qliksenseutil.exe you also want to update the configure-service.ps1 files to get the new database connection for the microservices (appsettings.json files) as described above.

Re: How to configure Qlik Sense to use a dedicated PostgreSQL database

Lokeshb31 — Tue, 04 Oct 2022 10:23:41 GMT

@Sebastian_Linser : I am not migrating DB to cloud, we just want to shift Qliksense repository from Default central node to another standalone separate database node managed by DBA team, both environments are on-premise, no cloud is involved here. Do we need to backup/restore all DBs in this case? Our Feb 2021 repository is 9.6(default) and the one we are shifting QSR is Postgres 12. What I did is, just modified Qliksenseutil connection string for QSR as per below screenshot and restarted all services. Its all working fine.

Could you please suggest what other steps I need to do?

Re: How to configure Qlik Sense to use a dedicated PostgreSQL database

Sebastian_Linser — Tue, 04 Oct 2022 15:00:52 GMT

The create script is identical for on prem and cloud. You still need to create the other DB's as a emtpy one. Migrating only if you fullfill the conditions in the last post. Otherwise they will be seeded by first Qlik service dispatcher start.

To see which additional connection strings you have to update check with that command

C:\Program Files\Qlik\Sense\dir /s Configure-Service.ps1

all you find need to be updated.

Re: How to configure Qlik Sense to use a dedicated PostgreSQL database

Lokeshb31 — Thu, 20 Oct 2022 05:15:38 GMT

@Sebastian_Linser : In this link its mentioned to update configure.ps1 with below command i.e. localhost DB credentials.

.\Configure-Service.ps1 localhost 4432 qliksenserepository Password123!

It should be .\Configure-Service.ps1 <newDBhost> <newDBport> qliksenserepository <newpassword>

Right?

I tried with .\Configure-Service.ps1 <newDBhost> <newDBport> qliksenserepository <newpassword>

But out of all Configure.ps1 locations ( found using dir /s Configure-Service.ps1), it threw error for below ones

D:\Program Files\Qlik\Sense\DataPrepService\install

D:\Program Files\Qlik\Sense\PrecedentsService\install

D:\Program Files\Qlik\Sense\MobilityRegistrarService\install

D:\Program Files\Qlik\Sense\NotifierService\install

Error message:

WARNING: Skiping the database initialization. No superuser or password specified.
Creating schema 'dataprep_service'.
Invoke-Query : Exception calling "Start" with "0" argument(s): "The system cannot find the file specified"
At D:\Program Files\Qlik\Sense\DataPrepService\install\install-utils\Postgres.ps1:56 char:12
+ $ret = Invoke-Query $command
+ ~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Invoke-Query], MethodInvocationException
+ FullyQualifiedErrorId : Win32Exception,Invoke-Query

Please help to address the issue.

Re: How to configure Qlik Sense to use a dedicated PostgreSQL database

Sebastian_Linser — Thu, 20 Oct 2022 06:44:02 GMT

are youy within the folder when you tun the Query?

so cd "D:\Program Files\Qlik\Sense\DataPrepService\install" and then running the command?

You can alternativly also run the command without the arguments and you will be promted to enter them.

This one here "Invoke-Query : Exception calling "Start" with "0" argument(s): "The system cannot find the file specified"" could be related to the postgres binary it searches. Which is not because you installed it on the D drive but because you have the folder 9.6 hardcoded in the original file.

Check for this line in Configure-Services.Ps1 and adjust it to your system

[string]$postgresHome = "$senseInstallPath\Repository\PostgreSQL\9.6",

e.g.

[string]$postgresHome = "$senseInstallPath\Repository\PostgreSQL\12.5",

Re: How to configure Qlik Sense to use a dedicated PostgreSQL database

Lokeshb31 — Thu, 20 Oct 2022 07:53:43 GMT

@Sebastian_Linser I edited path to [string]$postgresHome = "$senseInstallPath\Repository\PostgreSQL\12.5", but its not working, the obvious reason is, I dont have 12.5 installed on same host, its on another host. What changes I need to do in file?

Re: How to configure Qlik Sense to use a dedicated PostgreSQL database

Lokeshb31 — Thu, 20 Oct 2022 07:57:30 GMT

[CmdletBinding()] Param( [Parameter(Mandatory = $True)] [string]$databaseHost, [Parameter(Mandatory = $True)] [int]$databasePort, [Parameter(Mandatory = $True)] [string]$databaseUser, [Parameter(Mandatory = $True)] [string]$databaseUserPassword, [string]$databaseSuperUser, [string]$databaseSuperUserPassword, [string]$senseInstallPath = "$env:ProgramFiles\Qlik\Sense", [string]$senseDataPath = "$env:ProgramData\Qlik\Sense", [string]$postgresHome = "$senseInstallPath\Repository\PostgreSQL\9.6", [string]$hostname = "localhost", [string]$serviceName = "DataPrepService", [string]$databaseName = "SenseServices", [string]$databaseSchema = "dataprep_service", [string]$paramsLocation = "$senseDataPath\$serviceName" ) $ErrorActionPreference = "Stop"; $contentLocation = $PSScriptRoot . $contentLocation/install-utils/Json.ps1 . $contentLocation/install-utils/Postgres.ps1 . $contentLocation/install-utils/Encoding.ps1 . $contentLocation/install-utils/Parameters.ps1 . $contentLocation/install-utils/Security.ps1 # Set postgres execution environment $global:psqlPath = "$postgresHome/bin/psql.exe" $global:pgPassPath = "$contentLocation/pgpass.conf" $global:postgresPort = $databasePort $global:postgresHost = $databaseHost $global:postgresDB = $databaseName Write-Output "DataPrepService configuration started." try { if ($databaseSuperUser -and ($databaseSuperUserPassword -or $databaseHost -eq "localhost")) { Write-Output "Database superuser specified, initializing the database." # Set postgres user context $global:postgresUser = $databaseSuperUser Write-PgPassFile $databaseSuperUserPassword # Create database in external postgres instance Set-Database $databaseName $databaseSuperUser # Create service user accounts with CREATE privilege in database Set-ServiceUser $databaseUser $databaseUserPassword $databaseName Write-Output "Database initialization successful." } else { Write-Warning "Skiping the database initialization. No superuser or password specified." } # Reconfigure postgres user context $global:postgresUser = $databaseUser Write-PgPassFile $databaseUserPassword # Create database schema Set-Schema $databaseSchema $databaseUser # Read default appsettings $appsettings = Get-Content "$contentLocation\..\appsettings.json" -raw | ConvertFrom-Json # Set config values $appsettings.Postgres.Host = $databaseHost $appsettings.Postgres.Port = $databasePort $appsettings.Postgres.Database = $databaseName $appsettings.Postgres.Schema = $databaseSchema $appsettings.Postgres.Password = $databaseUserPassword | Encrypt-WithMachineKey $appsettings.Postgres.Username = $databaseUser # Save modified appsettings Write-Output "Saving the modified settings." $json = $appsettings | ConvertTo-Json -Depth 10 | Format-Json Out-File -InputObject $json -FilePath "$contentLocation\..\appsettings.json" # Create a copy of the invocation parameters for the upgrade Write-Output "Exporting the copy of the invocation parameters." $params = $PSBoundParameters | Export-JsonParameters | Encrypt-WithMachineKey New-Item -ItemType Directory -Force -Path $paramsLocation | Out-Null Out-File -InputObject $params -FilePath "$paramsLocation\configure.params" -Force Write-Output "DataPrepService configuration successful." } finally { Remove-PgPassFile }

Above is Configure.pS1

Re: How to configure Qlik Sense to use a dedicated PostgreSQL database

adamb14 — Mon, 30 Oct 2023 03:16:13 GMT

Hi,

While doing this can you also go from a Different PostgreSQL DB version to another? (e.g. 9.6 to 14.8)

Thanks.

Re: How to configure Qlik Sense to use a dedicated PostgreSQL database

Sonja_Bauernfeind — Mon, 30 Oct 2023 09:44:18 GMT

Hello @adamb14

I assume you intend to set up a new standalone database on a higher version (so, 14.8) and then restore the 9.6 database from your previous bundled PostgreSQL?

That is usually how PostgreSQL upgrades are done between major versions (install a new major version, restore the previous one), so the answer would be yes. If, however, you are also upgrading Qlik Sense at the same time we recommend staggering the procedure: Move to an identical version of PostgreSQL first, then upgrade Qlik Sense, then upgrade the database, taking the appropriate backups along the way.

You may also want to look into the Qlik PostgreSQL installer, which not only unbundles Qlik Sense from the PostgreSQL database but can also install a standalone DB. General "upgrade standalone PostgreSQL" instructions that go hand in hand with Qlik can be found here: Qlik Sense Enterprise on Windows: How To Upgrade Standalone PostgreSQL.

All the best,
Sonja

Re: How to configure Qlik Sense to use a dedicated PostgreSQL database

Luck — Wed, 10 Jan 2024 13:43:32 GMT

Hi,

Is this article still valid if I would like to just change port of the dedicated postgresql?
(example from port 4432 to 4433)

Regards,

Lucky

Re: How to configure Qlik Sense to use a dedicated PostgreSQL database

Sonja_Bauernfeind — Wed, 10 Jan 2024 14:21:30 GMT

Hello @Luck

The article is still accurate, yes. And you can change the port of your standalone PostgreSQL environment. If you have more detailed questions on how to achieve this, please post your question in the Qlik Sense Management and Deployment board.

All the best,
Sonja

Re: How to configure Qlik Sense to use a dedicated PostgreSQL database

TusharDev_Qlik — Wed, 07 Feb 2024 08:01:58 GMT

@Sebastian_Linser @Sonja_Bauernfeind

Remove Database Service dependency and Postgres Server

Can this step be performed post successful verification to change existing local install PostgreSQL database to dedicated server. This will help to rollback to previous setup in case we face any issues post implementation.

Re: How to configure Qlik Sense to use a dedicated PostgreSQL database

Sonja_Bauernfeind — Thu, 08 Feb 2024 12:54:13 GMT

Hello @TusharDev_Qlik

I can confirm that this step should be performed after you have successfully verified your setup. Will clarify this in the text above, thank you for getting in touch!

All the best,
Sonja

Re: How to configure Qlik Sense to use a dedicated PostgreSQL database

TusharDev_Qlik — Fri, 09 Feb 2024 05:12:30 GMT

@Sonja_Bauernfeind Thank you for your prompt response.

Could you please include details rollback steps(how to restore databases configurations to the previous version) in the above document which will complete the entire process.

Re: How to configure Qlik Sense to use a dedicated PostgreSQL database

Sonja_Bauernfeind — Fri, 09 Feb 2024 10:40:07 GMT

Hello @TusharDev_Qlik

A "rollback" by itself does not exist for this process. To move back to a bundled PostgreSQL instance, Qlik Sense Enterprise on Windows would need to be uninstalled and reinstalled on the identical Sense version and the database restored.

All the best,
Sonja

Re: How to configure Qlik Sense to use a dedicated PostgreSQL database

TusharDev_Qlik — Mon, 12 Feb 2024 05:49:37 GMT

@Sonja_Bauernfeind By rollback I mean if we repoint all databases to Localhost again before uninstalling PostgreSQL from Central, the system will be restored to the previous setup automatically isn't it?