Qlik Community

QlikView Scripting

Discussion Board for collaboration on QlikView Scripting.

Announcements

Breathe easy -- you now have more time to plan your next steps with Qlik!
QlikView 11.2 Extended Support is now valid through December 31, 2020. Click here for more information.

Not applicable

Incremental load

Hello,

I want to know about incremental load.

For an example:

1. I have made an "RA.qvw" with all my scripting in (getting data from my DB) - And Stored all my tables into QVDs

2. I have loaded QVD into an RB.qvw - with nothing done (QVD optimized load)

3. I have binary loaded RB.qvw into RC.qvw (loads really really fast) 1 sec or so for 25mil rows

My question is, how do i do incremental reload? and where do i do it?

I would say full load from RA.qvw and Incremental load from RB.qvw??

Please help!

Thanks in advanced

Tags (4)
11 Replies

Re: Incremental load

Hi,

     Do the increment load at RA level. - this will not take much time.

     Do a full load on RB - this might take time for full load.

     And full load on RC - This will not take much time

Regards,

Kaushik Solanki

jagannalla
Valued Contributor III

Re: Incremental load

Please find the attachment for Incremental load procedure.

It may help you.

Not applicable

Re: Incremental load

Can you explain why?

Why full load on RB?

So basically i wanna do incremental load from DB to RA - full load from RA to RB (why not incremental)?

Not applicable

Re: Incremental load

Thanks i have seen this - But im not sure i totally understand it. Where would i insert these scripts? In RA??

Re: Incremental load

Hi,

     See the reason for doing incremental at RA level is, the load on your database will be less cause it will send (or QlikView will fetch ) only the incremental data.

     Full load on RB is needed cause i believe that you are creating the datamodel at RB level and thus needs to have full data.

     Full load on RC is you load full data from RB. and thus will be fast, so doesnt need incremetal here.

egards,

Kaushik Solanki

Not applicable

Re: Incremental load

Thank you! But how do i do incremantal load (insert,update) ? Ive seen the script being posted, but i cant quite figure out how it will see updated rows.

For an example:

If i have a DimProduct which contains

ProductKey, ProductID, ProductName

1, 1, ok

2, 1, ok

3, 2, hej

4, 3, heeeeee

If i do incremental load on insert i just want to check if ProductKey exists in my QVW

but what i have an update row eg like

ProductKey, ProductID, ProductName

1, 1, UPLYLSA

2, 1, ok

3, 2, hej

4, 3, heeeeee

Re: Incremental load

Hi,

     Its always better that you have a field called LastUpdated, which will be a timestamp, which will tell us when the row is updated.

     Incremental will become very hard without LastUpdated field.

     Once you have that field the methods shown in PPT will work.

Regards,

Kaushik Solanki    

Not applicable

Re: Incremental load

Ahh - so i need a lastupdatedfield. But thats just not very often you have that in Dimensions. But thanks for the answer!

Last questions.

Where do you wanna put the script shown in PPT?

And should it be for each table?

Re: Incremental load

Hi,

     The lastupdated doesnt need to be a dimension, it will only help for incremental.

     The script shown in the PPT will be put in RA qvw as a script.

     Give a little efforts in searching the community about incremental load on qlikview, this will give you so many different scenarios.

Regards,

Kaushik Solanki