Qlik Community

QlikView Scripting

Discussion Board for collaboration on QlikView Scripting.

Announcements

Breathe easy -- you now have more time to plan your next steps with Qlik!
QlikView 11.2 Extended Support is now valid through December 31, 2020. Click here for more information.

andriesb
Not applicable

mapping load : from resident or qvd : best practice

To add different information extracted from an already saved qvd, I have to create 5 "mapping load"  's.

What is better and faster:

do a single load and then load the 5 different mappings information from resident memory

or

only load the mapping info 5 times

in code:

tempList:

Load *

from source.qvd (qvd)

mapping load

...

resident templist

(...5 times)

or

5 times:

mapping Load

field1,

field2

from source.qvd (qvd)

I could imagine loading only 5 times the mapping would result better in memory usage but would be slower to execute.

Any ideas/ experience to share on this?

7 Replies
jagan
Not applicable

mapping load : from resident or qvd : best practice

Hi,

First load the QVD and then using Resident load the mapping tables.  This has better performance than loading the QVD 5 times.  Resident Load is faster than loading data from QVD.

tempList:

Load *

from source.qvd (qvd)

mapping load

...

resident templist

(...5 times)

Regards,

Jagan.

andriesb
Not applicable

mapping load : from resident or qvd : best practice

and according to memory usage ? Loading all 5 milj. records versus only the information I actually needs for the mapping?

Loading from memory will always be faster, but the extra memory requered could be a drawback.

Speed is not always the first priority, as a 'reload' could be sceduled during night time.

amien
Not applicable

Re: mapping load : from resident or qvd : best practice

"Resident Load is faster than loading data from QVD."

is this really always the case? i just tried loading a QVD and than a resident load on the same table. QVD is faster.

rwunderlich
Not applicable

Re: mapping load : from resident or qvd : best practice

No, it's not always the case. In many cases loading from a QVD is faster than resident.

-Rob

Not applicable

Re: mapping load : from resident or qvd : best practice

Rob is it fair to say that an optimized QVD load will generally always be faster than a Resident load, whereas a QVD load that is unoptimized is not?

rwunderlich
Not applicable

Re: mapping load : from resident or qvd : best practice

I'm not sure that is correct. More to the point, I think it's rarely relevant because most of the time, we are doing operations in a Resident load that would not qualify for an optimized load.

Like:

LOAD max(Date) from ....

which suprisingly, is usually faster from a QVD even thought it's not optimized.

-Rob

phaneendra_kunc
Not applicable

Re: mapping load : from resident or qvd : best practice

I agree with Rob..QVD load is faster when compared to Resident....

One scenario i have is 30 mill records table needs to be reload twice with manipulations. so with my test i came to know that resident is taking longer than normal QVD load.

So we came up with a approach where First Load statement data will be stored into a temporary QVD and again the next LOAD statement will pick up this temp QVD instead of a resident..and does its job quicker..