Incremental load problem

[#1]

// step 1. Load a QVD file

NoConcatenate

        Temp_Decrypt:

        Load

        *

        From $(vFullFileName)(qvd);

//step 2. Load latest data from Database

NoConcatenate

        Temp_Incremental:

        LOAD

       *

        SQL Select *FROM vResTable

        where TimeModified <= '2013-09-13';

// step 3: Concatenate for incremental load

Concatenate (Temp_Incremental)

        LOAD

       *

        Resident Temp_Decrypt: // with resident table

        where not Exists (ID);

//step 4: store

STORE Temp_Incremental into $(vFullFileName);

The problem is

in step 5 it stores only data retrieved in step2 i.e. it is not concatenating with data retrieved in step 1. I do not have option to avoid step 1 and directly use the QVD file used in step3:

If I avoid step1. and directly use QVD file in step 3 then everything works fine.

[sasikanth]

Hai Ramya  I am sending one doc refer it ......this may  help to solve  your problem...

[rwunderlich]

"If I avoid step1. and directly use QVD file in step 3 then everything works fine."

Correct. That is the way it's supposed to work. You should load the QVD data only after loadin ghte updated rows from the DB.

-Rob

[#4]

Hello Rob,

could you provide me any QVW for incremental load for fact table.

Regards,

Ram

[kaushiknsolanki]

Hi,

     You can go on

     http://robwunderlich.com/

     And in download section you can download the cookbook, which has a good numbers of example.

Regards,

Kaushik Solanki

[amars]

Hi Ram,

Try like below, this should work

//step 1. Load latest data from Database

NoConcatenate

        Temp_Incremental:

        LOAD

       *

        SQL Select *FROM vResTable

        where TimeModified <= '2013-09-13';

// step 2. Load all data except data loaded previously from the QVD file

Concatenate (Temp_Incremental)
        Load

        *

        From $(vFullFileName)(qvd)

        where not Exists (ID);

//step 3: store

STORE Temp_Incremental into $(vFullFileName);

Thanks...

[#7]

Hi Rob,

My data is encrypted in QVD file so I can not directly use data from QVD file. I need to bring it to cache and decrypt and then concatenate it. That's why I have step 1 where i decrypt data.

My question is

isn't it possible to concatenate two resident tables?

what's difference does it make whether data is coming from QVD or Resident table in step 3?

the other way I made it make it work by dumping my decrypted data in another QVD file and use that QVD file in step 3. but that way...i am exposing my encrypted data for a brief moment which i want to avoid.

[#8]

Hi Michael,

I can not avoid step 1 where i load my decrypted data from QVD file in resident table. My data is encrypted in QVD file and i can not perform concatenation on that.

I have omitted the decrypt function name from my code but you can make out from then name of the resident table.

I want to do the concatenation process with my resident table which has decrypted data. Is there any limitation of Qlikview that it can not do concatenation for incremental load with resident table.

Saurabh

[rwunderlich · Accepted solution]

You can concatenate however you want. The problem you are having is in step 3:

where not Exists (ID)

All the ID values were loaded in step 1, therefore the test will never be true. You need to make a copy of ID and test against that.

In Step 2:

LOAD *, ID as ID2...

In Step 3:

where not Exists (ID2,ID)

Before the STORE

DROP FIELD ID2;

-Rob

http://masterssummit.com

http://robwunderlich.com

[#10]

It works.

Thanks

but why did you drop Field..why is it needed?