I'd like to know if it is possible to segment a data set that an application uses to improve performance.
We are working on a single application that will use a predefined data set, but the user base that will use the data is perfectly segmented by a given attribute within the data. Since the data set is large and consumes lots of memory on the server, searching and other functions take time if the user base is operating on the entire data set. Breaking this set down to each individual user's piece speeds up the response time tremendously.
For example, imagine the data set is geographical in nature, and the records are each assigned to a State. There are 1 Billion total records in the set. For sake of argument, say the records are spread evenly across each state with 20,000,000 per state. There are 50 total users, each of which is responsible for a state. If each user has to operate across all 1 Billion records, then it is much slower than if each of them can operate on only the 20 Million records they each care about. They all would use the same Qlik Sense app, but the data each of them cares about is only a portion of the entire set.
Is there some way to segment the data by an attribute, in this example State, in order to improve the performance? Does segmenting the data using security accomplish this? What would be the best approach to optimize this type of application/data?
Thanks!
