incremental load loading whole data

[jayati_shrivast]

Hi all,

I have created an incremental load, but it is loading the whole data when I reload.

Following is the script I am using for creating the incremental load:

LastUpdatedDate:
LOAD

max("DATE") as MaxDate

FROM [lib://qvd/book.qvd]
(qvd);

Let varThisExec = Date(Now());

Let varLastExec = peek('MaxDate', 0, 'LastUpdatedDate');

incremented_table:

LOAD
NAME,
"DATE",
ID
FROM [lib://BOOK/Book1.xlsx]
(ooxml, embedded labels, table is Sheet1)

Where "DATE" >= '$(varLastExec)';

Concatenate

LOAD

"NAME",

"DATE",

"ID"

from [lib://qvd/book.qvd]
(qvd)
where not Exists(ID);
Inner Join

LOAD
"ID"

FROM [lib://BOOK/Book1.xlsx]
(ooxml, embedded labels, table is Sheet1);
if '$(ScriptErrorCount)' = 0 then

Store incremented_table into [lib://qvd/book.qvd]
(qvd);

Let varLastExec = '$(varThisExec)';

endif
Exit Script;

pls help me.

Thanks

[punitpopli]

Hi 

The initial steps to get max date from the existing data is correct.

Post this you should create a IF statement which will check if new date is greater than date in the existing data then execute below script else exit script

and in the execution statements you should code like

Load * from new file;

Concatenate

Load * from OldFile;

Hope this helps.

[jayati_shrivast]

Thanks for the info. can u provide me an example with if statement, it will help me in implementing it more clearly.