Advanced Qlik Sense System Monitoring

Mario_Petre · Feb 28, 2025 2:36:37 AM

Chapters:

01:33 - Why use Zabbix
02:35 - Architecture for demo
03:41 - Downloading the installer
04:36 - Installing Zabbix Server
08:37 - Installing the Zabbix agent
12:17 - Applying Qlik specific templates
14:28 - Reviewing Qlik-specific Dashboards
16:49 - Configuration details
18:42 - How to create a dashboard
20:30 - Q&A: Can Zabbix run on Windows?
21:16 - Q&A: Is Zabbix supported by Qlik?
21:36 - Q&A: Can this monitor data capacity?
22:45 - Q&A: Can the Zabbix agents affect performance?
23:20 - Q&A: Can it monitor bookmark size?
24:02 - Q&A: Can this monitor amount of data being used?
24:19 - Q&A: Can this monitor sheets, and objects in apps?
24:49 - Q&A: Is there a similar tool for Cloud?
25:36 - Q&A: Would this work with QlikView?
26:11 - Q&A: Does this read the app data?
26:26 - Q&A: Can this help measure how long to open an app?

Content

Chapters:
Environment overview
Zabbix Server set-up
Clone the Zabbix docker repository
Setting up environment variables
Re-link compose.yaml to our preferred compose file
Logging in for the first time
Zabbix Agent installation on Windows Server
Zabbix Server Configuration
Adding the first Server
Importing Qlik Sense Enterprise for Windows templates
Linking templates to hosts
Engine Healthcheck Monitoring with HTTP Agent example
Steps to configure a new HTTP Agent for QSE Health monitoring
Defining the Virtual Proxy prefix for Zabbix HTTP Agent
Resources & Links

Environment overview

The environment being demonstrated in this article consists of one Central Node and Two Worker Nodes. Worker 1 is a Consumption node where both Development and Production apps are allowed. Worker 2 is a dedicated Scheduler Worker node where all reloads will be directed. Central Node is acting as a Scheduler Manager.

Zabbix Server set-up

The Zabbix Monitoring appliance can be downloaded and configured in a number of ways, including direct install on a Linux server, OVF templates and self-hosting via Docker or Kubernetes. In this example we will be using Docker. We assume you have a working docker engine running on a server or your local machine. Docker Desktop is a great way to experiment with these images and evaluate whether Zabbix fits in your organisation.

Clone the Zabbix docker repository

This will include all necessary files to get started, including docker compose stack definitions supporting different base images, features and databases, such as MySQL or PostgreSQL. In our example, we will invoke one of the existing Docker compose files which will use PostgreSQL as our database engine.

Source: https://www.zabbix.com/documentation/current/en/manual/installation/containers#docker-compose

git clone https://github.com/zabbix/zabbix-docker.git

Setting up environment variables

Here you can modify environment variables as needed, to change things like the Stack / Composition name, default ports and many other settings supported by Zabbix.

cd ./zabbix-docker/env_vars
ls -la #to list all hidden files (.dotfiles)
nano .env_web

In this file, we will change the value for ZBX_SERVER_NAME to something else, like "Qlik STT - Monitoring". Save the changes and we are ready to start up Zabbix Server.

Re-link compose.yaml to our preferred compose file

./zabbix-docker folder contains many different docker compose templates, either using public images or locally built (latest and local tags).

You can run your chosen base image and database version with:

docker compose -f compose-file.yaml up -d && docker compose logs -f --since 1m

Or unlink and re-create the symbolic link to compose.yaml, which enables managing the stack without specifying a compose file. Run the following commands inside the zabbix-docker folder to use the latest Ubuntu-based image with PostgreSQL database:

unlink compose.yaml
ln -s ./docker-compose_v3_ubuntu_pgsql_latest.yaml compose.yaml
Start the Zabbix stack in detached mode with docker compose up -d

If you skip the -d flag, the Docker stack will start and your command line will be connected to the log output for all containers. The stack will stop if you exit this mode with CTRL+C or by closing the terminal session. Detached mode will run the stack in background. You can still connect to the live log output, pull logs from history, manage the stack state or tear it down using docker compose down.

Pro tip: you will be using docker compose commands often when working with Docker. You can create an alias in most shells to a short-hand, such as "dc = docker compose". This will still accept all following verbs, such as start|stop|restart|up|down|logs and all following flags. docker compose up -d && docker compose logs -f --since 1m would become dc up -d && dc logs -f --since 1m.

Logging in for the first time

By default, the Zabbix Web GUI will be exposed on ports 80/443
Using tools like Portainer makes Docker stack management easier

Use the IP address of your Docker host: http://IPADDRESS or https://IPADDRESS.

The Zabbix server stack can be hosted behind a Reverse Proxy.

The default username is Admin and the default password is zabbix. They are case sensitive.

Zabbix Agent installation on Windows Server

Download link: https://www.zabbix.com/download_agents, in this case download the Windows installer MSI.

Run the installer .msi
Leave components unchanged
Hostname = your machine hostname, we will have to use the same hostname when adding a Host in Zabbix Server.
Zabbix server IP/DNS: IP address or DNS name of your Zabbix Server
Agent listening port, the same port will be used when when adding a Host in Zabbix Server.
Enable "Add agent location to the PATH" for convenience in the command line
Finish installation

Zabbix Server Configuration

Adding the first Server

After Agent is installed, in Zabbix go to Data Collection > Hosts and click on Create host in the top right-hand corner. Provide details like hostname and port to connect to the Agent, a display name and adjust any other parameters. You can join clusters with Host groups. This makes navigating Zabbix easier.

Importing Qlik Sense Enterprise for Windows templates

In the Zabbix Web GUI, navigate to Data Collection > Templates and click on the Import button in the top right-hand corner. You can find the templates file at the following download link:

LINK to zabbix templates

Linking templates to hosts

Once you have added all your hosts to the Data Collection section, we can link all Qlik Sense servers in a cluster using the same templates. Zabbix will automatically populate metrics where these performance counters are found. From Data Collection > Hosts, select all your Qlik Sense servers and click on "Mass update". In the dialog that comes up, select the "Link templates" checkbox. Here you can link/replace/unlink templates across many servers in bulk.

Select "Link" and click on the "Select" button. This new panel will let us search for Template groups and make linking a bit easier. The Template Group we provided contains 4 individual templates.

Fig 2: Mass update panel Fig 2: Mass update panel

Fig 3: Search for Template Group Fig 3: Search for Template Group

Once you Select and Update on the main panel, all selected Hosts will receive all items contained in the templates, and populate all graphs and Dashboards automatically.

To review your data, navigate to Monitoring > Hosts and click on the "Dashboards" or "Graphs" link for any node, here is the default view when all Qlik Sense templates are linked to a node:

Fig 5: Repository Service metrics - Example

Engine Healthcheck Monitoring with HTTP Agent example

We will query the Engine Healthcheck end-point on QlikServer3 (our consumer node) and extract usage metrics from by parsing the JSON output.

Steps to configure a new HTTP Agent for QSE Health monitoring

We will be using a new Anonymous Access Virtual Proxy set up on each node. This Virtual Proxy will only Balance on the node it represents, to ensure we extract meaningful metrics from the Engine and we won't be load-balanced by the Proxy service across multiple nodes. There won't be a way to determine which node is responding, without looking at DevTools in your browser. You can also use Header or Certificate authentication in the HTTP Agent configuration.

Once the Virtual Proxy is configured with Anonymous Only access, we can use this new prefix to configure our HTTP Agent in Zabbix.

Defining the Virtual Proxy prefix for Zabbix HTTP Agent

In the Zabbix web GUI, go to Data collection > Hosts. Click on any of your hosts. On tabs at the top of the pop-up, click on Macros and click on the "Inherited and host macros" button. Once the list has loaded, search for the following Macro: {$VP_PREFIX}. This is set by default to "anon". Click on "Change" and set Macro value to your custom Virtual Proxy Prefix for Engine diagnostics, and click Update. The Virtual Proxy prefix will have to be changed on each node for the "Engine Performance via HTTP Agent" item to work. Alterantively, you can modify the MACRO value for the Template, this will replicate the changes across all nodes associated to this Template.

Fig 6: Changing Host Macros from Inherited values

To make this change at the Template level, go to Data collection > Templates. Search for the "Engine Performance via HTTP Agent" and click on the Template. Navigate to the Macros tab in the pop-up and add your Virtual Proxy Prefix here to make this the new default for your environment. No further changes to Node configuration are required at this point.

Fig 7: Changing Macros at the Template level

The Zabbix templates provided in this article contain the following Engine metric JSONParsers:

Memory: Allocated, Committed, Free, Total Physical
Calls, Selections
Saturation status (true/false)
Sessions: Active/Total
Users: Active/Total

These are the same performance counters that you can see in the Engine Health section in QMC.

Stay tuned to new releases of the Monitoring Templates. Feel free to customise these to your needs and share with the Community.

Resources & Links

Environment

Qlik Sense Enterprise on Windows

Advanced Qlik Sense System Monitoring