Solved: Preceding Load vs Resident Load and Their Inner Wo... - Qlik Community

joydipp1988 · ‎2016-02-17

Dear all,

"Preceding Load works much faster than Resident Load"-- till now I found this topic as a myth in QlikView. Tried to find out the reason but haven't got a clear answer till now. But after I came to know about "Symbol Tables and Bit Stuffed Pointers", I think inner workings of Preceding Load and Resident Load can be described by using Symbol Tables topic. So if someone describe the reason in light of their inner workings it will be very charming. Thanks in advance.

Regards,

Joy

tresesco · ‎2016-02-18

Hello Joydip,

Perhaps your perception:

Joydip Chakraborty wrote:

In case of Resident Load data gets copied into RAM for the second time ....

is not right. QV probably would not load it twice, rather fetch the already loaded (in RAM) data.

And your statement:

In preceding load data fetched from data sources only once and gone through some transformations on the fly in the preceding steps and then finally loaded into RAM. ..

is not probably accurate either. The data is already there in RAM when any operation is being performed on it. Preceding load is like working in a pipeline in a bottom-up approach. Some operation is performed on a record (Note: it is one-by-one record process) and in another operation (preceding load) it operates on the result of first operation.

Now I would comprehend Why Preceding Load is Faster (in General) like:

In preceding load, multiple operations are done in a sequence/pipeline record-by-record, so the QV engine would fetch (from the base address of RAM) the same record only once for multiple operations. Whereas, in resident load, the same record has to be fetched multiple times for multiple operations.

Thanks.

View solution in original post

Anonymous · ‎2016-02-18

might be helpful for you?

Resident load and precedding load

Preceding Load and Resident Load

Precedent Load vs Resident Load

jonathandienst · ‎2016-02-18

Preceding load is almost always faster than a resident load. That is not a myth.

Preceding load vs resident has absolutely nothing to do with the symbol tables. Henric Cronstoms' article is the most authoritative explanation

Preceding Load

Logic will get you from a to b. Imagination will take you everywhere. - A Einstein

tresesco · ‎2016-02-18

IMHO - you are not really in the right path.

Joydip Chakraborty wrote:

"Preceding Load works much faster than Resident Load"-- till now I found this topic as a myth in QlikView.........

Could you explain your finding that leads you say as a Myth? It has to be more with RAM rather than Symbol Table and Bit Stuffed Pointers. In short, Preceding Load - doesn't require an extra time table load whereas, Resident Load - does.

Though, it is not 100% right pair to be compared. There are cases where you can't replace Resident Load by Preceding Load but the reverse can.

joydipp1988 · ‎2016-02-18

Hi Tresesco,

Yes I know there has some definite cases where we can't use Preceding load and Resident load is the only solution. They are- Loading Cross Tabular Data, On top of Joining or Concatenation of two tables, On top of Wildcard load,

Order By statement on top of initial load etc. But I'm actually bothered about the inner workings of Resident Load. I'm sharing my idea here. Please share your views regarding that.

In case of Resident Load data gets copied into RAM for the second time and processed. So as we do resident load either we need to use NoConcatenate to keep the second copy of the data or we need to Drop the previously loaded Table. In that way more RAM gets occupied and reload time increases in case of Resident Load. But this isn't the case for Preceding Load. In preceding load data fetched from data sources only once and gone through some transformations on the fly in the preceding steps and then finally loaded into RAM. So Data occupy RAM only once and processed once only. In that way Preceding load is better choice than Resident Load. Please confirm whether my ideas are right or wrong. Thanks in advance.