how to handle historical data in Qliksense

[qliksensetel]

Hi,

I am new to qliksense and need some help in a requirement i have to complete.

The data load time for qvds is quite slow since we have to deal with 11 years of data and manage incremental/deleted load. What i am planning to do is to have 1 application which has say data for last 1 year & there is another application which has data since inception. Both applications have same field and structure the only difference being the timestamp. The Application having 11 years of data can be loaded once a day while the other can be updated after every 15 mins. What i want to know is that what are the best practices suggested for qliksense considering this scenario or what i am planning to do sounds the right approach?

[marcus_sommer]

If you are slicing the qvd to a year- or maybe a yearmonth-level you don't need to load all ones for your application with the current data.

Beside this I suggest to check if you really need all fields: Search Recipes | Qlikview Cookbook and because of your mention to timestamps if you could reduce the cardinality of some fields, see: The Importance Of Being Distinct.

- Marcus

[qliksensetel]

Hi Marcus,

Thanks for our response.  Can you please explain what you mean when you said " you don't need to load all ones for your application with the current data."

The difference between the 2 applications would be

APP1 : Data with Last update date > = 1st Jan 2016

APP2: Data for whole 11 years not having any range for dates.

Can you please suggest if the timestamp link you shared should be relevant here as well?

[marcus_sommer]

In any way there must be something how you could differ between the data for APP1 and APP2 and then using at least two qvd's and not a single one.

The meaning of my second paragraph was to check if the data for the qvd and the applications could be optimized. Depending on the data there could the data be reduced to 20% and less and this could lead to the situation that no splitting of the data is needed and either the load-times as well as UI performance is sufficient enough - therefore read the links carefully.

- Marcus