topic Re: Connectivity and Data Prep for Near-live Sheets (SCOM) in Connectivity & Data Prep

Connectivity and Data Prep for Near-live Sheets (SCOM)

ThyFere — Wed, 27 Aug 2025 03:20:55 GMT

I want to build the visualization sheets using SCOM data. As SCOM reporting sucks, we have decided to move the dashboards to Qlik Sense but yes, we will have to build it from the scratch. So sheets will include the Performance metrics of thousands of servers, etc. Stake holders want it to be near-live data.

I have already created the DB connections to SCOM OperationsManager and Data Warehouse DBs. Now, how can I create the near-live sheets. I haven't done it before. Need your advice and guidance.

Re: Connectivity and Data Prep for Near-live Sheets (SCOM)

hugo_andrade — Wed, 27 Aug 2025 15:46:31 GMT

Hi @ThyFere ,

First thing is to pull the data in. Then, you would be able to optimize to near real time.

The idea is to minimize the load duration, and trigger reloads very often.

To trigger reloads often, I'll refer to this amazing guide I stumbled upon on LinkedIn: https://www.datavoyagers.net/post/comparing-methods-to-reload-qlik-applications-in-qlik-cloud

It covers many different approaches on performing frequent reloads.

On reducing the load duration, you should focus on Incremental Load which is a technique used to update data by loading only new or changed records from a source, rather than reloading the entire dataset. The concept is covered as a best-practice on Qlik's help: https://help.qlik.com/en-US/sense/May2025/Subsystems/Hub/Content/Sense_Hub/LoadData/use-QVD-files-incremental-load.htm

Do you have a field on the data that represents the moment each row was last modified?

If this helps you, please give a like and mark as Solved.

Thanks

Re: Connectivity and Data Prep for Near-live Sheets (SCOM)

howdash — Wed, 27 Aug 2025 16:04:09 GMT

This sounds like you'll need the app to continuously be fed latest data. To setup something like this, you'll need to split your Qlik Sense app into two apps.

First App: Data Loader/Modeler

Have the first app load the data that you need and model it however you need it. The purpose of this app will be just to pull and model the data. It won't have any charts or anything else on the front-end.

Second App: The Front-End

Have the second app load the data from the first app through something called a binary load and use the loaded data to create tables, filters, and all the visualizations that you need in the second app.

Here's a good community post that shows an example of how to setup a data connection that you can use to binary load data from other apps.

Create Reload Tasks

Once both apps are built, create a reload task for the first app to have an event trigger that will reload first app when the second app finishes reloading.

Then also create a reload task for the second app to also have an event trigger that will reload the second app when the first app finishes reloading.

Lastly, manually kick off the reload task for the first app. The first app will load the data and when it's done, the second app will pull in loaded data from the first app.

When the second app finishes loading the data from the first app, it will signal to the first app that it's done loading so the first app will start reloading to pull newer data. And so on an so forth in an infinite loop.

This loop will continue until you stop it by disabling one of the tasks. Otherwise, you will have a continuous feeding of latest data into your app.