topic Re: QVD fragmentation in QlikView

QVD fragmentation

Wed, 10 Sep 2014 22:20:13 GMT

STORY:

I have QV app which does ETL process for another QV app. It produces a bunch of QVD files ~ 500 000 files.

I run the QVW from command line with different parameters in 8 parallel sessions.

Problem:

The QVD files are with fragmentation. If I load last created QVDs to a memory and store it into another file without any changes the difference is significant - let's say originally generated file is 1035 Kb; re-stored file - 14Kb.

Is this a bug or a feature?

You can find the example in attachment. You just need to reload QVW file and you will find re-stored file in QVD folder.

Thanks in advance for any help.

Re: QVD fragmentation

Thu, 11 Sep 2014 17:11:13 GMT

I've found where the problem is. It's not about QVD fragmentation. It's about QVD metadata. There is a <Lineage> tag in a QVD header which stores all SQL/Load statements from your script. The amount of data will grow if you have some Load statements in Loop cycle, which I do.

Next question is - is there any way to turn this Lineage tag off for all my QVDs?

Re: QVD fragmentation

Fri, 12 Sep 2014 14:43:46 GMT

Ok. Here is a workaround to rid of Lineage info on Server/Desktop from version 10SR3 to 11SR8 (tested):

Shutdown all QV services
Go to folder %APPDATA%\QlikTech\QlikView\
Edit Settings.ini with: Before the statement “CurrentQVConnect64Loc” add the “AllowDataLineage=-1” ( make sure this has the upper and lower letter cases)
Restart the QV services

For those who wants to reduce size of existing QVD files with big Lineage tag, you can use QVW apps from attachment:

Copy qvd-defrag.qwv into folder with the QVDs
Reload the QVW app

Have fun.