Today, I'd like to share with you our current App backup solution using Powershell and Qlik-Cli.

As you know, a production environment needs backups and Qlik Sense Enterprise does not offer such a thing by default so I decided to build a solution.

Using Qlik-Cli for Windows, I put in place a solution that will not only backup my published and unpublished apps but will also create a folder structure to match your streams.

The unpublished apps are backed up inside the "My Work" folder using the app owner's name as a directory name. This allows to easily find the app owner when performing a search in the folders.

Square brackets in the App's name will be removed because you cannot create a file with these characters in Windows.

The script is scheduled using the Windows Task Scheduler with the following command:

Here's the code:

$ErrorActionPreference = "Stop"

Try
{
# set export drive and directory
D:
cd D:\QlikBackups

# connect to the Qlik Server. Replace <ServerName> with your Qlik Sense hostname
Connect-Qlik -ComputerName <ServerName> -UseDefaultCredentials -TrustAllCerts

# Export each apps from the previously defined streams to qvf
Get-QlikStream | % {New-Item -Name ".\$($_.name)" -Force -ItemType 'Directory'} | foreach {Get-QlikApp -filter "stream.name eq '$($_.name)'"} | foreach {Export-QlikApp -skipdata -id $_.id -filename "$($_.stream.name)\$($_.name -replace '[[\]]','').qvf"}

# set the export folder to the My Work area for unpublished apps
cd "D:\QlikBackups\My Work"

# Create a folder structure for all the app owners
Get-QlikApp -full -filter "published eq False" | % {New-Item -Name ".\$($_.owner.userId)" -Force -ItemType 'Directory'} 

# Export each app into their folder
Get-QlikApp -full -filter "published eq False" | foreach {Export-QlikApp -skipdata -id $_.id -filename "$($_.owner.userId)\$($_.name -replace '[[\]]','').qvf"}
}
Catch # In case of error, send an email to the Qlik administrators
{
    $ErrorMessage = $_.Exception.Message
    $FailedItem = $_.Exception.ItemName
    $Time=Get-Date
    "Error: $FailedItem Message: $ErrorMessage Time: $Time" | out-file d:\log\QlikBackup.log -append
    
    $FailMailParams = @{
To = 'qlikadmin@MyCompany.com'
        From = 'MyQlikServer@MyCompany.com'
        Port = '25'
        SmtpServer = 'smtprelay.MyCompany.com'
        Subject = 'MyQlikServer Backup Failure'
        Body = "The backup failed. The error message was: $ErrorMessage"
        }

         Send-MailMessage @FailMailParams
}

My next steps are:

• Add timestamps to the filename
• Keep the most recent copy and Archive the rest.
• Add qvf file retention

Recently we’ve been adding a lot of content on our Qlik Sense Enterprise server. This combined with frequent reloads for near real-time data means our ArchivedLog folder is growing fast. With a 1 year retention in place, we’re currently sitting at 32gb. I expect we’ll double this by the end of the year. This may not seem like much but with the current disk space available in our SAN, it’s becoming a problem (There’s not just Qlik that’s growing rapidly).

I looked into multiple solutions such as having a daily/weekly script run to timestamp & zip files and keep them available for audit purposes but I still wanted to have my files available without restoring anything. That’s when I thought about using the almost forgotten feature, NTFS compression. This feature was once seen as bad because CPU power was much more limited than today and we’re now also running on SSDs. Both of these factors have made file compression a non issue.

I simply navigated to my ArchivedLog directory and enabled file compression (may take a while…). All the files were compressed and I was now using 60% less space than before with no impact whatsoever on my task durations.

Please note that you should be attempting this on your Dev environment first AND that this is better done outside business hours as there is a chance you will encounter file locking while the compression is running for the first time.

I like this feature as I have nothing to script and test and I can always turn this off without any issues. None of my app reloads, including the monitoring apps that read log files have increased in load times.

Let me know if your savings are as good or better as mine!

Environment

• Qlik Sense Enterprise November 2021

The information in this article is provided as-is and to be used at own discretion. Depending on tool(s) used, customization(s), and/or other factors ongoing support on the solution below may not be provided by Qlik Support.

Related Content

Refer to this page for a written guide on enabling file compression.

A very famous data visualisation is the plot of the signal from Pulsar CP 1919. The reason it is famed is not least because it featured on the cover of Joy Division's album Unknown Pleasures.

This app has the data from that plot (or an approximation of it) rendered in Sense.

Unknown Pleasures

The data set can be found here:
https://bl.ocks.org/borgar/raw/31c1e476b8e92a11d7e9/pulsar.csv