incremental reload

ali_hijazi — Wed, 27 Jul 2011 20:38:37 GMT

Hello

I'm implementing incremental reload on 2 tables, Header and Details that are associated by a column named INVKEY

The scenario that I have is Insert only on the original datasource

for the very first time I load all data from both tables and store them in qvd files

on the next reload I'm getting the max CreateDate ( a field in which the date at which the record was created is stored) from the QVD file

then I get records from database whose CreateDate >= the max(CreateDate)

then concatenate the retrieved result with the qvd file as follows:

Concatenate(InvoiceHeader)

LOAD * from $(vQVDPath)InvoiceHeader.qvd(qvd)

where not(Exists(INVKey));//INVKey is the primary key

Store InvoiceHeader into $(vQVDPath)InvoiceHeader.qvd(qvd);

and the Details as follows:

Concatenate(InvoiceDetails)

LOAD * from $(vQVDPath)InvoiceDetails.qvd(qvd)

where not(Exists(INVKey));//INVKey is the primary key

store InvoiceDetails into $(vQVDPath)InvoiceDetails.qvd(qvd);

suppose the last reload was made on 27/7/2011 morning at 9 and there are records already saved to datasource on 27/7/2011 at 8

then in the after noon new details were inserted in the datasource for existing header records

how can I make sure that these new records entered in the afternoon are saved to the qvd knowing that i put the condition on the details table during concatenat "where not(Exists(INVKey));" to make sure that the records records saved on 27/7/2011 at 8 are not duplicated?

topic incremental reload in QlikView

incremental reload