Incremental Load

[#1]

Hi,

I have created a QVD file and I would like to know how I do incremental load as I am getting new files daily. Below is my QVD script

Events:

LOAD [Customer Nr],

     Date(Date,'YYYY-MM-DD') as Date_EVENTS,

     Time as Event_Time,

     [Event Nr],

     [Event Name],

     Product,

     [Device Nr],

     [Reason Description],

     Username,

     [Full Name],

     Location as [Event Location],

     1 as EventFlag,

     Name,

     Cell,

     Email,

     Region as Events_Region,

     [Location No],

     [Location No] & '_' & Date(Date,'YYYY-MM-DD') as EVENTS_JOINKEY

FROM

[\\03rnb-qlkvw01\QVData\EventsData\SAUserEvent*.csv]

(txt, codepage is 1252, embedded labels, delimiter is ',', msq);

STORE Events INTO \\03rnb-qlkvw01\D\QVData\Elija's_QVD's\Events.qvd (QVD);

[sivarajs]

Check the attached file.

Schedule your qvw to run on daily basis.

[its_anandrjs]

It depends on the table structure and behavior if it has primary key then use primary key field for incremental load. If it has max updated date then use that field for the incremental otherwise you need to find the key max updated.

Hope this helps

Thanks & Regards

[Yousef_Amarneh]

You have to move old file on daily basis to archive file and then use concatenate. or if the files names contain date then you can handle it in your script:

LET vToday = Date(Today());

Events:

load *

from \\03rnb-qlkvw01\D\QVData\Elija's_QVD's\Events.qvd (qvd);

concatenate(Events)

LOAD [Customer Nr],

     Date(Date,'YYYY-MM-DD') as Date_EVENTS,

     Time as Event_Time,

     [Event Nr],

     [Event Name],

     Product,

     [Device Nr],

     [Reason Description],

     Username,

     [Full Name],

     Location as [Event Location],

     1 as EventFlag,

     Name,

     Cell,

     Email,

     Region as Events_Region,

     [Location No],

     [Location No] & '_' & Date(Date,'YYYY-MM-DD') as EVENTS_JOINKEY

FROM

[\\03rnb-qlkvw01\QVData\EventsData\SAUserEvent$(vToday).csv]

(txt, codepage is 1252, embedded labels, delimiter is ',', msq);