Incremental load problem

[#1]

// step 1. Load a QVD file

NoConcatenate

        Temp_Decrypt:

        Load

        *

        From $(vFullFileName)(qvd);

//step 2. Load latest data from Database

NoConcatenate

        Temp_Incremental:

        LOAD

       *

        SQL Select *FROM vResTable

        where TimeModified <= '2013-09-13';

// step 3: Concatenate for incremental load

Concatenate (Temp_Incremental)

        LOAD

       *

        Resident Temp_Decrypt: // with resident table

        where not Exists (ID);

//step 4: store

STORE Temp_Incremental into $(vFullFileName);

The problem is

in step 5 it stores only data retrieved in step2 i.e. it is not concatenating with data retrieved in step 1. I do not have option to avoid step 1 and directly use the QVD file used in step3:

If I avoid step1. and directly use QVD file in step 3 then everything works fine.

[rwunderlich]

Field ID2 was created just as a place to collect the ID values loaded in Step 2. ID2 is only needed for the WHERE NOT EXISTS test in Step3, as a filtering mechanism. It is not needed after that and should not be kept in the final data model.

-Rob