How much data size could QVD handle?

[Anonymous]

How much data size could QVD handle?

Is there any data size limitation?

I have a data set with 200 million rows and 50 columns. Is it too big for Qlik QVD? Is it too big for Qlik sense to load?

Thanks!

[vlad_komarov]

There is no limitation for data to be stored in QVD(s). The only possible limitation is your PC's system resources  (HD size and Memory size).

There were some reports that QS saves large QVD files unproportionally slower than the smaller ones, but I believe it's also related to HD's condition (fragmentation, etc) and available HD space....

Regards,

Vladimir

[rwunderlich · Accepted solution]

A QVD can handle that amount of data no problem,

Although a single QVD can handle all that data there are situations where is advantageous to break the table into multiple QVDs, segmented by a period or dimensional value, such as a Month or Region. Reasons for doing so include:

1. When QVDs need to be updated by incremental load. It is usually faster to update a few smaller QVD files than a single large file.

2. When the QVD consumer may want a limited time period or segment of data. If you only want the last 12 months -- it is faster to load the specific 12 monthly QVDs than loading and filtering a single QVD that holds 5 years.

3. Rolling off old data. If you want to keep a rolling 5 years, it can be faster to delete old monthly files instead of trimming data from a single file.

The Month case used above is only an example. You can of course use any segmentation that makes sense. If your reload times are acceptable with a single QVD, by all means stick with a single QVD as it is easier to script with.

-Rob