This content has been marked as final. Show 4 replies
I need to do incremental load every month, however there might be some data duplicate with previous upload, may I know can I remove the duplicate rows?
** Upload ID was given to every incremental upload, therefore the row will not be exactly the same due to the Upload ID.
The following is my code:
LOAD max(UploadID) as previousID
LET vID = peek('previousID')+1;
'$(vID)' as UploadID,
now(1) as "Date Uploaded"
(ooxml, embedded labels, table is Sheet1);
STORE [Dataset] into [lib://QVD/Dataset.QVD] (qvd);
Load * ,' ' as junk resident [Dataset];
LOAD * from [lib://QVD/DatasetMain.QVD](qvd);
Store [Dataset] into [lib://QVD/DatasetMain.QVD];