You probably knows the incremental load example in Rob Wunderlich's cook book (if not: http://robwunderlich.com/downloads/ ), I am refererring to that file.
It also contains a lot of comments, so it's worth having a look at.
The approach uses
a) a Modification Date
b) a unique Key per row
the script finds the max modification date from the historical data in the qvd.
Then it starts building the data table with loading newer data from the data source.
It then concatenates the existing historical data from the qvd, but only if the unique Key not already exists
(this is the place with "where not exists"). Thus we can not only add data to the historical data, but also modify existing rows (because only the row with newer ModDate is kept, the historical data with same unique key will not be read).
You probably knows the incremental load example in Rob Wunderlich's cook book (if not: http://robwunderlich.com/downloads/ ), I am refererring to that file.
It also contains a lot of comments, so it's worth having a look at.
The approach uses
a) a Modification Date
b) a unique Key per row
the script finds the max modification date from the historical data in the qvd.
Then it starts building the data table with loading newer data from the data source.
It then concatenates the existing historical data from the qvd, but only if the unique Key not already exists
(this is the place with "where not exists"). Thus we can not only add data to the historical data, but also modify existing rows (because only the row with newer ModDate is kept, the historical data with same unique key will not be read).
