Append incremental load

[smilingjohn]

Hi Al, 

Please find the attcahment of the QVD and the sample data of excel attached. 

I am trying to have a incremental load by appendin the the new data to the old one . To do so i have scripted the incremental load as attached  but in this it is replacing the values ? I want to keep all the values as it is on top of that it should keep appending the new values . 

If you look at my data screenshot below  the first fetch .

ID's (A5 and A6) created on 19Nov2020 with status Open  and was updated on 29Nov2020 

I have stored this inofrmation QVD  after next reloading now the ID's (A5 and A6) have been updated on 1Dec20 and 02Dec20 with different status . 

and my output is just replacing them ? How can manage to keep all the previous records and then append the new one also . 

I want to append all the data  where Updated date is greater than previous update date.  

Can please someone help me in achieving this

Thanks in Advance

[edwin · Accepted solution]

try instead of just using ID in PKEY, use ID & '|' & DATE(Updated,'D-MMM-YY') and use the same format in your not exist check

where not Exists(PKEY,ID & '|' & DATE(Updated,'D-MMM-YY') )

.  the reason it is excluding prior dates is bec the ID already exists in the new set.  but if it is combination of ID and UPDATE date then you preserve prior dates.