Qlik Community

Qlik Sense App Development

Discussion board where members can learn more about Qlik Sense App Development and Usage.

Not applicable

Segment Data Set for an App to Speed Performance?

I'd like to know if it is possible to segment a data set that an application uses to improve performance.

We are working on a single application that will use a predefined data set, but the user base that will use the data is perfectly segmented by a given attribute within the data.  Since the data set is large and consumes lots of memory on the server, searching and other functions take time if the user base is operating on the entire data set.  Breaking this set down to each individual user's piece speeds up the response time tremendously.

For example, imagine the data set is geographical in nature, and the records are each assigned to a State.  There are 1 Billion total records in the set.  For sake of argument, say the records are spread evenly across each state with 20,000,000 per state.  There are 50 total users, each of which is responsible for a state.  If each user has to operate across all 1 Billion records, then it is much slower than if each of them can operate on only the 20 Million records they each care about.  They all would use the same Qlik Sense app, but the data each of them cares about is only a portion of the entire set.

Is there some way to segment the data by an attribute, in this example State, in order to improve the performance?  Does segmenting the data using security accomplish this?  What would be the best approach to optimize this type of application/data?

Thanks!

3 Replies
allardos
Contributor

Re: Segment Data Set for an App to Speed Performance?

Using section access would probably help you in this case; because all 50 users don't work with the entire dataset to start with. I expect Qlik's internal memory management to efficiently cope with these smaller portions.


Really splitting up the dataset into multiple smaller applications is not available (yet) in Qlik Sense. The functionality you describe is available in QlikView, and is called "Loop and Reduce" that generated similar dashboards with only a part of the dataset, based on the values in a specific field.

I want support for this feature in Qlik myself. See the following thread:

Is there "Loop and Reduce" equivalent functionality available in Qlik Sense server?

allardos
Contributor

Re: Segment Data Set for an App to Speed Performance?

Found a (good) solution yet?

I have asked some people today during the "Visualize Your World" Virtual Forum event; didn't result in a satisfying solution just yet. But I did get some additional input, see this thread: https://community.qlik.com/thread/165488

Not applicable

Re: Segment Data Set for an App to Speed Performance?

Is this function now available (segment the data set to certain users of the qlik app)?