topic Incremental Load in QlikView

Incremental Load

Mon, 31 Mar 2014 10:12:23 GMT

Hi,

I have created a QVD file and I would like to know how I do incremental load as I am getting new files daily. Below is my QVD script

Events:

LOAD [Customer Nr],

Date(Date,'YYYY-MM-DD') as Date_EVENTS,

Time as Event_Time,

[Event Nr],

[Event Name],

Product,

[Device Nr],

[Reason Description],

Username,

[Full Name],

Location as [Event Location],

1 as EventFlag,

Name,

Cell,

Email,

Region as Events_Region,

[Location No],

[Location No] & '_' & Date(Date,'YYYY-MM-DD') as EVENTS_JOINKEY

FROM

[\\03rnb-qlkvw01\QVData\EventsData\SAUserEvent*.csv]

(txt, codepage is 1252, embedded labels, delimiter is ',', msq);

STORE Events INTO \\03rnb-qlkvw01\D\QVData\Elija's_QVD's\Events.qvd (QVD);

Re: Incremental Load

sivarajs — Mon, 31 Mar 2014 10:17:10 GMT

Check the attached file.

Schedule your qvw to run on daily basis.

Re: Incremental Load

its_anandrjs — Mon, 31 Mar 2014 10:17:59 GMT

It depends on the table structure and behavior if it has primary key then use primary key field for incremental load. If it has max updated date then use that field for the incremental otherwise you need to find the key max updated.

Hope this helps

Thanks & Regards

Re: Incremental Load

Yousef_Amarneh — Mon, 31 Mar 2014 10:25:03 GMT

You have to move old file on daily basis to archive file and then use concatenate. or if the files names contain date then you can handle it in your script:

LET vToday = Date(Today());

Events:

load *

from \\03rnb-qlkvw01\D\QVData\Elija's_QVD's\Events.qvd (qvd);

concatenate(Events)

LOAD [Customer Nr],

Date(Date,'YYYY-MM-DD') as Date_EVENTS,

Time as Event_Time,

[Event Nr],

[Event Name],

Product,

[Device Nr],

[Reason Description],

Username,

[Full Name],

Location as [Event Location],

1 as EventFlag,

Name,

Cell,

Email,

Region as Events_Region,

[Location No],

[Location No] & '_' & Date(Date,'YYYY-MM-DD') as EVENTS_JOINKEY

FROM

[\\03rnb-qlkvw01\QVData\EventsData\SAUserEvent$(vToday).csv]

(txt, codepage is 1252, embedded labels, delimiter is ',', msq);