I'm currently doing an incremental load (with update)... but I face an issue with the exists function. When I try to load the data from existing QVD and use the exists function to remove the updated records, some of the records are missing. For example:-
QVD: 121,317 records
Updated data: 82 records
Final QVD with concatenated data: 31507 (!!!)
Here is the sample script:-
LOAD SalesOrderID,
SalesOrderDetailID,
CarrierTrackingNumber,
OrderQty,
ProductID,
SpecialOfferID,
UnitPrice,
UnitPriceDiscount,
LineTotal,
rowguid,
ModifiedDate;
SQL SELECT *
FROM Test.Sales.SalesOrderDetail
where SalesOrderID in (select distinct SalesOrderID FROM Test.Sales.SalesOrderHeader where [ModifiedDate]='2004-07-31 00:00:00.000');
