Incremental load for insert update delete

Wed, 05 Jun 2013 09:16:19 GMT

Dear All,

for Incremental load

a single table with a unique field, last modify time

1, find last load time from QVD

2, select from DB where modify time > last load time

3, concatenate QVD, and newly-retrieved DATA using not-exists-uniquekey <-- This Combination contains the updated and inserted record, not yet the deleted ones.

4, compare unique key between newly-concatenated Data and DB by Inner join. <-- excluding the deleted records.

for 4, any other mothod to optimise this process? because if there are thousands of records,

Database still consumes significant time --- loop to compare from 1st to last row?

I have only one idea for it:

1, only Allow modification for records generated few days ago

Could anyone give me some idea? thanksssssssssssssssssssssssssssssssssssssssssssssss

Calvin

topic Incremental load for insert update delete in QlikView

Incremental load for insert update delete