How to reduce the data in a QVD

[Anonymous]

Hi,

We have a QVD which will have only 39 months of data and our goal is to have the data from the 40 month dropped of the QVD every month.

Our current QVD has more than 53 million of rows and 247 columns. We tried using the WHERE clause to filter the rows but it took around 5 hrs. To get the new QVD done, also we tried doing an INNER JOIN with only the rows we want but it took almost 3 hrs.

We are looking for other options so we want to hear from the community more ideas on how to get this done.

Thanks to all.

[#2]

Hi,

I would advise you split your QVD into multiple QVD's with the same name and suffix with the YearMonth. For example if you have one big ledger table, you can split that into multiple QVD's withe the names Ledger2010Jan and etc.

While loading your data into QVD you can pick the QVD's you need to load the required month's data into the memory.

Hope this helps.

Regards,

Murali