Skip to main content
Announcements
Have questions about Qlik Connect? Join us live on April 10th, at 11 AM ET: SIGN UP NOW
cancel
Showing results for 
Search instead for 
Did you mean: 
Not applicable

Incremental load

Hello,

I want to know about incremental load.

For an example:

1. I have made an "RA.qvw" with all my scripting in (getting data from my DB) - And Stored all my tables into QVDs

2. I have loaded QVD into an RB.qvw - with nothing done (QVD optimized load)

3. I have binary loaded RB.qvw into RC.qvw (loads really really fast) 1 sec or so for 25mil rows

My question is, how do i do incremental reload? and where do i do it?

I would say full load from RA.qvw and Incremental load from RB.qvw??

Please help!

Thanks in advanced

11 Replies
kaushiknsolanki
Partner Ambassador/MVP
Partner Ambassador/MVP

Hi,

     Do the increment load at RA level. - this will not take much time.

     Do a full load on RB - this might take time for full load.

     And full load on RC - This will not take much time

Regards,

Kaushik Solanki

Please remember to hit the 'Like' button and for helpful answers and resolutions, click on the 'Accept As Solution' button. Cheers!
jagannalla
Partner - Specialist III
Partner - Specialist III

Please find the attachment for Incremental load procedure.

It may help you.

Not applicable
Author

Can you explain why?

Why full load on RB?

So basically i wanna do incremental load from DB to RA - full load from RA to RB (why not incremental)?

Not applicable
Author

Thanks i have seen this - But im not sure i totally understand it. Where would i insert these scripts? In RA??

kaushiknsolanki
Partner Ambassador/MVP
Partner Ambassador/MVP

Hi,

     See the reason for doing incremental at RA level is, the load on your database will be less cause it will send (or QlikView will fetch ) only the incremental data.

     Full load on RB is needed cause i believe that you are creating the datamodel at RB level and thus needs to have full data.

     Full load on RC is you load full data from RB. and thus will be fast, so doesnt need incremetal here.

egards,

Kaushik Solanki

Please remember to hit the 'Like' button and for helpful answers and resolutions, click on the 'Accept As Solution' button. Cheers!
Not applicable
Author

Thank you! But how do i do incremantal load (insert,update) ? Ive seen the script being posted, but i cant quite figure out how it will see updated rows.

For an example:

If i have a DimProduct which contains

ProductKey, ProductID, ProductName

1, 1, ok

2, 1, ok

3, 2, hej

4, 3, heeeeee

If i do incremental load on insert i just want to check if ProductKey exists in my QVW

but what i have an update row eg like

ProductKey, ProductID, ProductName

1, 1, UPLYLSA

2, 1, ok

3, 2, hej

4, 3, heeeeee

kaushiknsolanki
Partner Ambassador/MVP
Partner Ambassador/MVP

Hi,

     Its always better that you have a field called LastUpdated, which will be a timestamp, which will tell us when the row is updated.

     Incremental will become very hard without LastUpdated field.

     Once you have that field the methods shown in PPT will work.

Regards,

Kaushik Solanki    

Please remember to hit the 'Like' button and for helpful answers and resolutions, click on the 'Accept As Solution' button. Cheers!
Not applicable
Author

Ahh - so i need a lastupdatedfield. But thats just not very often you have that in Dimensions. But thanks for the answer!

Last questions.

Where do you wanna put the script shown in PPT?

And should it be for each table?

kaushiknsolanki
Partner Ambassador/MVP
Partner Ambassador/MVP

Hi,

     The lastupdated doesnt need to be a dimension, it will only help for incremental.

     The script shown in the PPT will be put in RA qvw as a script.

     Give a little efforts in searching the community about incremental load on qlikview, this will give you so many different scenarios.

Regards,

Kaushik Solanki

Please remember to hit the 'Like' button and for helpful answers and resolutions, click on the 'Accept As Solution' button. Cheers!