Qlik Community

Qlik Product Innovation Blog

Learn about what's new across all of the products in our growing Qlik product portfolio.

phc
New Contributor III

Leave Data Where It Is

The Big Data ‘hype’ may have died down at this point but for many of our customers big data is still a really big deal.  Today, companies have a wide range of tools at their disposal for managing and processing big data but one aspect of working with big data remains a concern – that is how to make big data accessible, relevant, and interactive to every business user.  Most big data systems are great for processing big data in batch jobs or for supporting the quantitative elite but are just too slow to query in real time and work with interactively.  It is true that in some cases, this pain can be reduced but at great financial cost making it difficult to deliver the full potential of your big data investments across the entire business. 


Qlik On-Demand App Generation to the rescue!

Over the past few years, Qlik has worked closely with some of our largest customers to develop techniques that provide an interactive user experience from Big Data so that every user can benefit from these investments.  And, the best part, this technique works just as well on Qlik Sense as it does on QlikVIew.

As a simple example, imagine a telco company that has data from every touch point between every cell-phone and every cell-tower.  (That’s big data!)  A customer calls the telco call-center asking for help with a connectivity issue on their phone that they experienced last Tuesday.  The phone rep doesn’t really need ALL of the data in the big data store to do that analysis but they do need to be able to work interactively with the data that is relevant to the caller in real time so that they can help them.

On-Demand App Generation (ODAG) provides the ability for a user to first select a subset of data that they are interested in from a Big Data lake and then generates a detailed app with the relevant data for the user to explore interactively. 

In our example, the phone rep might select the caller’s phone number and all of the cell towers within a wide radius around the area where the caller was traveling last Tuesday.  Qlik On-Demand App Generation will then spawn a customized instance of the analysis app with just the data that is needed to help this customer.  Since the customized version of the analysis app is now in-memory, Qlik is able to deliver a tailor made highly interactive experience.  Why is this important?  Because this allows the phone-rep to work with that customer in real-time solving their problem and improving customer service. 

We will share more specifics about how to work with On Demand App Generation in both Qlik Sense and QlikView in the future. Stay Tuned!

Qlik On-Demand App Generation was actually introduced last June after working with a number of large customers to develop the technique.  Over the past year we have worked to provide more a more integrated solution which is what you will be seeing this June.


What’s Cooking @ Qlik

This information and Qlik‘s strategy and possible future developments are subject to change and may be changed by Qlik at any time for any reason without notice. This information is provided without a warranty of any kind.  The information contained here may not be copied, distributed, or otherwise shared with any third party.

ODAG is an incredible tool to have in your Big Data toolbox but there is still room for improvement.  In the future, our goal is to take this a step further delivering the best of both worlds – a direct connection back to a ‘live’ big data store and a highly interactive user experience that delivers the Associative Experience.

With On-Demand App Generation, we solve the performance concerns of working with Big Data but in order to request a different ‘slice’ of the data, users need to move back to the selection app and start over.  And, of course, working on the entire data lake is not possible using this model.

At Qonnections recently we were able to get a preview of just how this is expected to work in the future.

In addition to continuing to offer the On-Demand App Generation approach to Big Data, Qlik is working toward a solution currently referred to as Associative Big Data Indexing.  Imagine a future with the full associative experience on top of a big data lake without moving the data.  This model involves a parallel array of indexing engines optimized for Qlik style associative queries and speed.


Here is what that might look like in the future...

bigdataindex.jpg


Data can remain located in the cloud, on premises, or even a combination.  And, the Associative Big Data Index can be reused across multiple apps so everyone across the organization can gain the benefits and insights in your big data investments.  We look forward to sharing more about On Demand App Generation and Associative Big Data Indexing in the future.

4 Comments
marchoctober
Contributor

Hi PeggySue! Thank for this post. Please share more detail technical and non technical info regarding ODAG, as for last time it was very funny to see how platform runs when some of our devs started to run a test of ODAG on 70GB Sense app file

It'll be very helpful if you share the mechanism on how ODAG is initiating, reducing and selecting the data etc., to avoid users go ahead straight and test it immediately on their big apps!

155 Views
stantrolav
Contributor II

I read nothing new. It's a chain of documents. We already used this mechanism in our applications in order to decrease volume of data that would be used in calculations and increase performance of user application by that.

0 Likes
155 Views
Employee
Employee

Please be careful in terminology.

There are two different ways to work with the huge amount of data: chained applications and ODAG.

They are works differently.

Chained application require to define the chains at the design stage. So chained apps is the real example of "Guided Analytics" in terms of predefined selections of data. These predefined way of selections define the possible chains of applications. And they are defined not by end-user. They are defined by application's architect.

From technical perspective all apps in chain need to be fully prepared before any user will use them: required set of data loaded, all visual objects created (including the objects (buttons with actions) to navigate between the applications in chains).

ODAG works differently. This approach allow user to use ANY combinations of data selections. These selections use can made in usual Qlik app. After this Qlik by itself generate a new application with new load script (based on some template). So technically Qlik generate new load script, perform it and load data related to current user's selections.

From business perspective user have full freedom to don't have any limits on combinations of selections.

155 Views
phc
New Contributor III

marchoctober‌ - With the June release, we will have a new detailed section of our help site dedicated to on-demand apps.  Once we go live (currently expected to be near the end of June), I would love your feedback.

155 Views