Incremental load

[#1]

Hello,

I want to know about incremental load.

For an example:

1. I have made an "RA.qvw" with all my scripting in (getting data from my DB) - And Stored all my tables into QVDs

2. I have loaded QVD into an RB.qvw - with nothing done (QVD optimized load)

3. I have binary loaded RB.qvw into RC.qvw (loads really really fast) 1 sec or so for 25mil rows

My question is, how do i do incremental reload? and where do i do it?

I would say full load from RA.qvw and Incremental load from RB.qvw??

Please help!

Thanks in advanced

[kaushiknsolanki]

Hi,

     Do the increment load at RA level. - this will not take much time.

     Do a full load on RB - this might take time for full load.

     And full load on RC - This will not take much time

Regards,

Kaushik Solanki

[jagannalla]

Please find the attachment for Incremental load procedure.

It may help you.

[#4]

Can you explain why?

Why full load on RB?

So basically i wanna do incremental load from DB to RA - full load from RA to RB (why not incremental)?

[#5]

Thanks i have seen this - But im not sure i totally understand it. Where would i insert these scripts? In RA??

[kaushiknsolanki]

Hi,

     See the reason for doing incremental at RA level is, the load on your database will be less cause it will send (or QlikView will fetch ) only the incremental data.

     Full load on RB is needed cause i believe that you are creating the datamodel at RB level and thus needs to have full data.

     Full load on RC is you load full data from RB. and thus will be fast, so doesnt need incremetal here.

egards,

Kaushik Solanki

[#7]

Thank you! But how do i do incremantal load (insert,update) ? Ive seen the script being posted, but i cant quite figure out how it will see updated rows.

For an example:

If i have a DimProduct which contains

ProductKey, ProductID, ProductName

1, 1, ok

2, 1, ok

3, 2, hej

4, 3, heeeeee

If i do incremental load on insert i just want to check if ProductKey exists in my QVW

but what i have an update row eg like

ProductKey, ProductID, ProductName

1, 1, UPLYLSA

2, 1, ok

3, 2, hej

4, 3, heeeeee

[kaushiknsolanki]

Hi,

     Its always better that you have a field called LastUpdated, which will be a timestamp, which will tell us when the row is updated.

     Incremental will become very hard without LastUpdated field.

     Once you have that field the methods shown in PPT will work.

Regards,

Kaushik Solanki    

[#9]

Ahh - so i need a lastupdatedfield. But thats just not very often you have that in Dimensions. But thanks for the answer!

Last questions.

Where do you wanna put the script shown in PPT?

And should it be for each table?

[kaushiknsolanki]

Hi,

     The lastupdated doesnt need to be a dimension, it will only help for incremental.

     The script shown in the PPT will be put in RA qvw as a script.

     Give a little efforts in searching the community about incremental load on qlikview, this will give you so many different scenarios.

Regards,

Kaushik Solanki