topic Re: optimize and non-optimze load in qvd in QlikView

optimize and non-optimze load in qvd

Thu, 26 Jan 2012 23:01:30 GMT

Hello can any one help me to under stand the difference b/w

optimize and non-optimze load in qvd .

1:-which one is better and why ?

2:- one one scnerio for each .

Thanks .

optimize and non-optimze load in qvd

Oleg_Troyansky — Fri, 27 Jan 2012 00:04:49 GMT

Optimized load is much faster and therefore preferable, especially for larger data sets.

Optimized loads are possible if no transformations (calculations) are made during the load, and no (almost) filtering is done in a WHERE clause.

The only acceptable filter is using function Exists:

WHERE Exists(Field)

So for example this load can't be optimized:

Load * From ... WHERE Country = 'US'

but this load can:

Load * From ... WHERE EXISTS(Country)

================================

On the other hand, the only time you want the load not to be optimized, is if you load a Mapping table from a QVD file. In this case, Optimized load doesn't quite work, so you want to make the load non-optimized deliberately.

cheers,

Oleg

optimize and non-optimze load in qvd

Anonymous — Sat, 05 May 2012 14:32:17 GMT

Hi There,

For more info regarding optimized and unoptimized qvds, please read the artical of Rob Wunderlich:

http://qlikviewnotes.blogspot.com/2008/05/when-less-data-means-more-ram.html

Cheers,

iSam

optimize and non-optimze load in qvd

jonathandienst — Sat, 05 May 2012 18:50:24 GMT

I believe that it is no longer necessary to make mapping table loads not optimised (v11 and on).

Regards

Jonathan

optimize and non-optimze load in qvd

Thu, 10 May 2012 00:59:11 GMT

Hello,

On one hand we say that we should have try and have all the calculations in the script itself rather than putting them in the Dashboard & on the the other hand if we do that the load would no longer be Optimized. What to do then, how to have the expressions in the script itself and still make the load Optimized

Regards

Re: optimize and non-optimze load in qvd

Thu, 20 Sep 2012 13:46:33 GMT

I just found that when doing a

Load * From ...

the load will not be optimized. Only when I do a load with all fieldnames listed an optimized load is performed:

Load Field1, Field2, Field3 From ....

QV 11, I don't know if this behaviour is new in this release

Re: optimize and non-optimze load in qvd

Tue, 30 Jul 2013 09:18:28 GMT

hi,

The article by Rob states like this.

"Consider a QVD file with one million rows. The QVD was loaded from a database and contains two fields:

aNum – unique integers, 1M unique values.
aDate – dates distributed equally throughout 2000-2003, 1,460 unique values.

QV stores each of these values as integers, occupying 4 bytes of RAM each. Nice and compact."

Does this mean that the values in a QVD are stored as integers? Can anyone explain how the data is stored internally in a QVD?

Thanks,

Maria

Re: optimize and non-optimze load in qvd

Tue, 30 Jul 2013 09:30:43 GMT

hi,

Can anyone please confirm this point? Is it necessary to make mapping table loads not optimised from version 11?

Thanks,

Maria

Re: optimize and non-optimze load in qvd

Tue, 30 Jul 2013 09:38:39 GMT

No, it is not necessary to have unopt. load for mapping Loads..for sure

Re: optimize and non-optimze load in qvd

Anonymous — Fri, 14 Mar 2014 08:10:18 GMT

Hi Maria

i was also wondering how data is stored on the QVD did you get any answer?

thanks a lot

Re: optimize and non-optimze load in qvd

Tue, 04 Oct 2016 07:23:44 GMT

Hi Jonathan,

QVD is storing unique data with pointer to the actual values. Every field has its own table and pointers to that value. Pointers are bit stuffed and they are used to lookup actual values. Result of this kind of keeping is better compression. This compression is done every time when load script is running.

Does anyone know why when loading data in loops with concatenation without where or transformation is not optimized?

Re: optimize and non-optimze load in qvd

Wed, 05 Oct 2016 21:56:56 GMT

Hello,

Can anyone tell me the definition of what is Optimized and what is Unoptimized ?(Only definition)

Re: optimize and non-optimze load in qvd

Thu, 06 Oct 2016 06:21:22 GMT

Hello Bob,

Optimized load is reading qvd into application without filters (where clause or transformations), non-optimized is with filters.

Re: optimize and non-optimze load in qvd

Thu, 24 Nov 2016 17:29:29 GMT

Hi Lav,

I'm looking for answers on your question. I know its 2016, but pl share if you found answer to it.

Thanks, Shravan.

Re: optimize and non-optimze load in qvd

Tue, 28 Feb 2017 11:22:35 GMT

Hi Stravan K S,

If you use a multi-tier architecture, you can enjoy the speed of optimized load, since you have made all of the aggregation previously and a store into qvd command, and after when you would use this qvd-s you wouldn't have to use a command which would break the opt load.

Dan