Use of Resident Load ?

[pradeep92]

Hi,

I am having a data like ,

Product:

LOAD ProductID,

     [Product name],

     Sales,

     [Modified date],

     [Product country],

     Export,

     Sales*ProductID as Ex  // made own transformation

FROM

Incremental.qvd;

I know resident load can load data from previous existing data,make transformations and calculations etc.. but when is the situation to use it bcz i know i can directly make calculations in the loaded data itself ?

[vishsaggi]

Few examples here:

Resident load?

[MK9885]

Use Preceding load if you are joining same tables.

If you joining different tables you can use Resident load.

Resident load is performance heavy. This would read your table twice.

[sunny_talwar · Accepted solution]

Take a look at this scenario....

LOAD Dim,

     Sales,

     Quantity,

     Cost

     Sales * Quantity as TotalRevenue,

     Sales*Quantity - Cost as Margin,

     (Sales*Quantity - Cost)/(Sales*Quantity) as Margin%

FROM ...

So, technically the above can be performed in the same load or can be done in Preceding or Resident Load. Note that the calculations are easy enough to performed in a single load, but that is not always the case.. At times the calculations are so complex that it might always make sense to break it down into preceding loads to make your life easy and make the code readable.

LOAD *,

     Margin / TotalRevenue as Margin%

LOAD *,

     TotalRevenue - Cost as Margin;

LOAD Dim,

     Sales,

     Quantity,

     Cost,

     Sales * Quantity as TotalRevenue

FROM ....

[pradeep92]

Great ! So to make the calculations and code in the readable format , we make use of Preceding and resident load. I do know preceding load is faster compared to Resident but what is the exact situation to make use of both the loads?

[sunny_talwar]

The above is not the only reason to use Preceding or Resident loads. And also, they are not always the same in all the cases... One specific case where preceding load won't work, but resident load will is when you have to Order By clause. There can be other reasons such as performing aggregation in the script and still keeping the un-aggregated data.

Also, I don't think Preceding provides a huge improvement over Resident. I think it does to a point, but have read somewhere that having multiple preceding loads can actually be more slower than Resident.