Optimized .QVD Confusion

[#1]

Hello QlikCommunity,

I have been researching the best practices of .QVD use, and have been left with one question regarding the optimization of .QVD loads. When it is said that a .QVD is optimized, does this mean the .QVW that is creating the .QVD will create it more efficiently and quicker? Or does an optimized .QVD mean that the .QVW absorbing the .QVD's will do so more efficiently and quicker?

It was my assumption that an optimized .QVD was only quicker when being created (or loaded to incrementally). If this is the case, a .QVD that is recreated nightly would not necessarily benefit from optimization. But rather it would be better to move the data transformations to the .QVD so that the application .QVW does not have to do them.

Let me know if my thoughts/questions aren't clear.

Thank you for the help QlikCommunity!

-Chris

[rbecher]

Hi Chris,

I think your're mixing things here. There is no optimzed QVD file format (could be optimized in size by using smaller data types and/or fields with less cardinality althought) but there is an optimized QVD LOAD. Optimized load means the fastest way to load data from a QVD file into memory (and QVW).

- Ralf

[peter_turner · Accepted solution]

Hi Chris,

The optimised QVD is a method for loading the QVD into your QVW in the fastest way possible.

There are rules associated with this, and some script operations or conditions will stop the QVD from being loaded optimised. Loading from a QVD is quick anyway, but the optimised load dumps the entire file straight into RAM (as far as i know).

There are some posts / blogs on the community about the rules to allow an optimised load, i'll post a link if i find them.

[peter_turner]

There is a great post by Steve Dark which details information about the optimised QVD loading here

http://www.quickintelligence.co.uk/qlikview-optimised-qvd-loads/

[#5]

I think I am starting to understand more clearly. When creating the .QVD, you can have as many transformations, operations, or conditions as you want. But then when you absorb the .QVD into a .QVW you have to watch what transformations, operations, or conditions you are using for the load to continue to be optimized. Is this correct?

Thank you again for sharing your knowledge!

-Chris

[stevedark]

HI Chris,

You may want to take a look at my blog post on the subject:

http://www.quickintelligence.co.uk/qlikview-optimised-qvd-loads/

Steve

[mbm]

Chris,

Here is a good article explaining the ways to ensure your QVD loads are optimized from Steve Dark:

http://www.quickintelligence.co.uk/qlikview-optimised-qvd-loads/

[peter_turner]

Hi Chris,

Looks like Steve's post is very popular!

Yes your understanding is correct, so its best to try and get as much of the work done before you store to QVD.

Also we've found that loading an optimised QVD is faster than a resident table load.

[userid128223]

Hi

If you need calculations to be done after QVD is loaded as in in optimized fashion. How do you handle that. In the article it says do calculations and other modification in qvd release stage vs reading stage. can some please explain this in a simple code.

[stevedark]

Hi,

You simply can't do calculations on the fields coming out of a QVD without making the load non-optimized.  If the load is quick enough anyway then don't worry about it too much.  On small tables it is not a problem.  If you have a significant number of rows of data in your QVD you will want to ensure that calculations are done when you extract the QVD rather than read it.

So to give you some code, you may want to work out a duration between two dates, like this:

Date(EndTime - StartTime, 'hh:mm:ss') as Duration,

Doing this when loading from a SQL table will make zero difference how long it takes to load.  Doing it when loading from QVD could make it up to 100 times slower.

Hope that helps,

Steve