Skip to main content
Announcements
Have questions about Qlik Connect? Join us live on April 10th, at 11 AM ET: SIGN UP NOW
cancel
Showing results for 
Search instead for 
Did you mean: 
lawrance
Creator II
Creator II

Exceeds QVD Size using concatenate

Hi,

I have four 11 GB QVD files when I try to concatenate four QVD files into Single QVD but my final QVD contains 200 GB.

Can anyone please provide solution why it takes 200 GB.

Thanks,

Lawrance A

3 Replies
johnca
Specialist
Specialist

What does the data model look like after the concatenation? Did it really make 1 table?

marcus_sommer

I assume that you expect that to merge 4 qvd with 11 GB should result in a single qvd with about 44 GB. But the data-handling from qlik isn't just adding the data like with text-files else they will be stored in symbol- and data-tables. This could end in a qvd-size which is significantly smaller as the simple summing of 44 GB but also like in your case with a qvd-size which is much larger - caused probably from larger pointers. How it worked could you see here: Symbol Tables and Bit-Stuffed Pointers - a deeper look behind the scenes.

Beside this: do you really mean GB and not MB? By MB I would be not sure if I would merge the qvd's - by GB I wouldn't think on merging them.

- Marcus

Oleg_Troyansky
Partner Ambassador/MVP
Partner Ambassador/MVP

In addition to everyone's input - you should look for a couple of things:

- a single field that has a high number of distinct values, perhaps a transaction ID or something like that. This field is likely using a lot of memory, both for symbols and for the index. Consider removing the field if you can or soing something to reduce its cardinality. Worst case - you can use Autonumber in your resulting App to save on the symbol space.

- numeric fields or dates that are being stored as Dual values and use more than 8 bytes of memory for each value.

You can easily find these problems using Rob Wunderlich's Document Analyzer - it's a free tool that you can download here: www.QlikViewCookbook.com.

We teach these and many other performance techniques at the Masters Summit for Qlik. Check our agenda and decide if it might be helpful for you.

Cheers,

Oleg Troyansky

Upgrade your Qlik skills with my book QlikView Your Business: An Expert Guide to Business Discovery with QlikView and Qlik Sense