topic Incremental load in QlikView

Incremental load

nikhilgarg — Sun, 30 Aug 2015 13:46:46 GMT

Hello,

Does for incremental load in our database, we must have a date field telling last updated date ?

And if we don't have last updated date field, then is there any way for doing incremental load?

Thanks

Re: Incremental load

swuehl — Sun, 30 Aug 2015 14:07:56 GMT

It will depend on which incremental load scenarios (add, update, delete records) you need to handle.

Then you need to identify a primary key in your source table, if this primary key is for example an OrderID and the OrderID will be incremented by new orders, you can pretty much translate the approach using a last updated date to your setting with OrderID ( ...WHERE OrderID > $(vMaxOrderIDFromQVD) ) [but not for all scenarios like updated records].

Re: Incremental load

nikhilgarg — Sun, 30 Aug 2015 14:15:22 GMT

Ohk,

U mean:

1: For Insert, apart from date field we can use primarykeyfield(Orderid) instead of date field to load newly added data.

2: For update, date field required

3: No date field required, We need to make inner join with primary key.

Re: Incremental load

swuehl — Sun, 30 Aug 2015 14:55:10 GMT

Mostly like that.

1. will work best if your primary key is somehow incremented by each new record and you can use a simple comparison in a WHERE clause to your DB LOAD statement (like I tried to show above).

It will be much more complicated / less efficient if your available primary keys are unique but somewhat randomly distributed across value range.