Hi,
We are working on dashboard where the requirement is to do the analysis of data for last 5 years. We are pulling the data from a fact table which has billions of records. We are doing pre-aggregation of this data month-wise and keeping the pre-aggregated data in QVD files. There is one QVD file for each month. However after pre-aggregation the number of records in QVD file is around 2 million for each month. This is resulting into huge amount of data when we load the QVD files for 60 months (5 years) into the memory.
Can anyone suggest how to improve the performance in such cases. What is the best approach when we have high volumes of data ? I have read some blogs about partial reload to load the data on fly. But i am not sure if it is a good practice to use.
Thanks,
Samir
