Skip to main content

Suggest an Idea

Vote for your favorite Qlik product ideas and add your own suggestions.

Announcements
Have questions about Qlik Connect? Join us live on April 10th, at 11 AM ET: SIGN UP NOW

Partial Reload as a standard option for a QMC Task

RadovanOresky
Partner Ambassador
Partner Ambassador

Partial Reload as a standard option for a QMC Task

The partial reload would be very beneficial for working with IoT or online data.

It enables to have almost real-time data in the user app without the need to reload the whole data model => requires fewer resources.

It can already be triggered with API, but should definitely be an option for a Task Trigger in QMC.

Along with this, there should also be an option to set a timeframe when an hourly Task will NOT be executed (skipped). For example, many companies run consolidation and cleanup procedures on their databases during the night, so it does not make sense to reload data in specific hours.

In the context of a partial reload that runs every minute, there needs to be a possibility to postpone a partial reload and run a full reload for the rest of the data model.

10 Comments
juraj_misina
Luminary Alumni
Luminary Alumni

I'd even love to have an option to decide in script and do either full or partial reload based on conditions.

RadovanOresky
Partner Ambassador
Partner Ambassador

Yes! That is an excellent idea.

There is an IsPartialReload() system function, so why not ReloadMode() function or variable, similar to how ErrorMode works.

John_Teichman
Former Employee
Former Employee

Hello @RadovanOresky,

Thanks for your idea. Going forward please use one, and only one, label per post. Please see the submission guidelines for more details.

thomasmaure
Luminary Alumni
Luminary Alumni

Hello 

I fully agree. Partial reload seems really a good way to reach 'low latency' dashboard without having to burn CPU by reloading the same data all the time. Currently it seems limited to user session , is "difficult" to activate and requires good understanding of the ADD / REPLACE / IsPartialReload() statements to function correctly.

However, I do not the scheduler as the best spot for this feature. Indeed, there are other techniques like application pipelines and QVD to optimize data load in the scheduler and thus reduce load duration. The scheduler role is not to deliver high frequency reload but more to maintain data consistency between all user sessions even in case of reload failure. 

I see a bigger interest directly in the dashboard to allow user to update the dashboard very rapidly without impacting other users (so limited to user sessions). As Qlik Sense engine is taking care of refreshing transparently charts anytime the application data are modified, it should definitely be possible to create an automatic refresh / reload mechanism based on partial reload to limit CPU impact... 

So I did some experiments to evaluate the potential 

First thing is that it is possible to activate it via a simple widget ( https://www.youtube.com/watch?v=_fgUBRndn30 ) but it still requires user action. 

Second thing was then to build a simple extension with an ON/OFF button and a timer to activate periodic partial reload . The reload is local to user session only and not STORE is possible, but this can be used to query new data from a rest API and update dashboards accordingly . 

 

In my example , the refresh period is decided at development time and cannot be modified after app is published, and of course , it require a specific security rule so that users get the right to launch the reload.  

This can be combined with scheduler driven reload to align periodically the different users sessions.  

If interested, I imagine I can share the extension code , but without any warranty of its value 🙂 

Regards

Thomas

SteveCarlsberg
Contributor
Contributor

Qlik have added a new 'merge' functionality to partial reloads in the February 2021 release: https://help.qlik.com/en-US/sense/February2021/Subsystems/Hub/Content/Sense_Hub/Scripting/ScriptPref...

So evidently they are actively developing partial reloads - if this is the case, why not add it as an option in QMC??

marpiotrowicz
Contributor III
Contributor III

It will be nice to have partial reload in QMC

Thomas_Hopp
Employee
Employee

Hi @RadovanOresky coming back to your idea from last year. We are looking into this as a potential improvement for Q3. It is under investigation and so to speak not possible to confirm at this point that we will be able to deliver it. But it is in our plan for this year if technically easily possible.

Best regards,

Thomas

Status changed to: Open - On Roadmap
Juha
Partner - Contributor
Partner - Contributor

Hi @Thomas_Hopp! Any updates on this issue? Does Q3 or this year look possible? We're eagerly waiting for this as it prevents us from using partial reloads at all because we do our reloads with QMC.

SaaraKorppi
Partner - Contributor II
Partner - Contributor II

Hi,

As noted above, QS November release allows partial reloads from QMC tasks, which could help us making app reloads faster.

Our understanding is that the Qlik Sense APIs have a couple of different ways of starting reloads. The way we are doing this is as the easiest and most straightforward way -  by sending an HTTP POST request to /qrs/app/{id}/reload . That causes Qlik Sense to check if a reload task for that app exists, create it if it doesn't exist, and then start it.

Another way is to do the above yourself using the task-related APIs in Qlik Sense.

The problem is that apparently only the latter now allows you to specify you want a partial reload. Hence are there any plans to include the first-mentioned approach into allowing partial reloads from QMC tasks?