Qlik Engine Health-Check Analysis (app included)

[RadovanOresky]

Monitoring current performance and availability of Qlik Sense site is not out-of-the-box feature, but can be easily achieved with native REST connector and requesting engine\healthcheck JSON API.

There are other tools and approaches, but since I already have the best BI tool available 😉, I found it reasonable to use it. All that is necessary is just:

1. download and import the attached app
2. create REST Data Connections and adjust the settings in the script
3. schedule app reload

This Qlik Sense app:

• loads engine\healthcheck JSON data from multiple nodes
• parses and consolidates it to two tables
• stores them to QVDs with all previous snapshots
• visualizes key metrics of engine utilization and availability in time

It's recommended to deploy and schedule this app on DEV or TEST site, pointing to all production nodes. Otherwise, there will be no data loaded in case the production engine (or the whole server) crashes.
(just FYI - DEV and TEST sites are included in the professional license subscription)

It's also recommended to read the official help site info about the engine API before you deploy this app.

After importing the app to a suitable site, follow these steps:

1. Decide on target folder for result QVD files, where parsed snapshot of JSON data will be stored. Create a folder Data Connection (if it doesn’t already exist) and set its Lib name to vQVDpath variable in Settings script section.

2. Create a REST Data Connection to each node:

URL: https://<server domain>/engine/healthcheck OR https://<node IP address>:4747/engine/healthcheck

Authentication Schema: Windows NTLM
User name & password: type root admin or service user credentials
check "Skip server certificate validation" if there is no valid certificate on target node

Query headers:
User-Agent: Windows
Cache-Control: no-cache

3. Adjust the INLINE table in the Settings section of the script according to your site

node - name of the node that will be shown for app’s user
restConnection - name of the REST connection that points to node’s engine/healthcheck
RAM - amount of node's memory in GB
CPU - number of node's logical processors

4. Schedule a reload task. The recommended app reload frequency is 5 minutes or less (tested reload duration on the 4-node site is 12 seconds). The script will return "N/A" in [isSaturated] field when the engine is not responding.

The current roadmap includes - script for email notifications, possibility to load only N days back, mapping app name to IDs, node comparisons.

PLEASE! If you make adjustments or additions to the script or front-end visualizations, please share them back on the Qlik Community.

Happy Qliking 🙂
-RADO-

[RadovanOresky]

Hi @santhoshkumar , I guess that you'll need to set this up correctly in the REST data connection settings. You will probably not use Win NTLM auth schema and query headers will have a different specification. I unfortunately never tried this.

Would be great if you could share the settings if you'll succeed so I update the document.

[iceman35]

Hello @RadovanOresky , thank you for your reply.

I managed to make it work for my central node but my rim nodes still get the same error.

The "Test connection" works fine though but the URL http://[my_server]/engine/healthcheck response behave differently between the central and the rim nodes.

On the rim nodes, the URL redirects to the Qlik login page.

Bastien

[RadovanOresky]

Hi @iceman35, 

Not easy to say, what could be the problem... Ideas:

- Do you have Qlik Sense Proxy service running on RIM node? (it is necessary as the healthcheck goes through the local proxy)

- Can you access the healthcheck API from the node, where Scheduler service is running? (I believe that ports 4243 and 4242 must be opened; check it by opening the healthcheck URL from the browser on a remote desktop of the Central and Scheduler node)

- Are you using the Qlik service account to authenticate the healthcheck REST connection, or some other? If you use a different user then make sure it has admin access to Rim nodes.

These are just some quick thoughts. Hope something will help or at least lead you to solution.

[iceman35]

Hi @RadovanOresky ,

Yes I have a Qlik Sense Proxy on every node.

The scheduler is running on the central node and I get the same behaviour when I try to access the healthcheck URL directly from this server (login page).

Regarding the authentication configuration on the virtual proxies in the QMC:

- Central node ==> Windows authentication pattern: Windows

- RIM nodes ==> Windows authentication pattern: Forms

[YANGXU]

Hi guys,

I'm definitely wrong. Can you help me out?
Attached are the error and configuration screens

Thank you

[RadovanOresky]

Hi @YANGXU 

I think, you just need to clear the empty lines in the inline definition table. Like this:

EngineNodes:
load * inline [
node, restConnection, RAM, CPU
MyNode, libEngineHC, 32, 8
];

The empty line in the inline table will be passed as a valid record with empty values, which is not what you want.

[sergeyko]

Hi @RadovanOresky  - This looks awesome. Can't wait to have it working in our environment.

That said, I am getting the same error as reported by @YANGXU above. Tried cleaning up the inline load as you suggested but it didn't help resolve the issue.

The REST connection seems to be working in general but the script runs into some issues with parsing the response from the Engine HC endpoint. Can you please help us investigate further?

[sergeyko]

@RadovanOresky & @YANGXU - I was able to resolve the issue. It turned out that I specified the required variables as Query Parameters and not Header Parameters in my REST connection. Once I moved both variables to the Header Parameters section, it resolved the error.

[jchacko_rxsense]

@RadovanOresky In this app, is it possible to capture RAM utilization by Streams and Users?
I would like to also capture,  node statistics by stream and user i.e. RAM usage per stream and user.

[RadovanOresky]

Hi @jchacko_rxsense,

Unfortunately no. This app loads data from the simple API call, that has only general info.

For this kind of detailed analysis you'll probably need the Telemetry Dashboard.