Qlik Community

QlikView Scripting

Discussion Board for collaboration on QlikView Scripting.

Announcements
QlikView Fans! We’d love to hear from you.
Share your QlikView feedback with the product team… Click here to participate in our 5-minute survey.
Rules, plus terms and conditions, can be found here.
Not applicable

Insert, Update, Delete Incremental Load Without Entire History

So basically we get a nightly feed which includes 7 days worth of updated/new data. Along with that we get a file including all deleted id's. My objective is to do an incremental load, insert and update, using my initial QVD (made form a "catch up" file) and the 7 day file we get nightly. I then need to use the deleted ID file to remove any records from the table I created in the incremental load. Since we don't get a full export I can't utilize the inner load of all IDs most people use.

What would be the best way to go about this?

1 Reply

Re: Insert, Update, Delete Incremental Load Without Entire History

Maybe like this:

// load deleted ID's in a temporary table

T1:

LOAD ID FROM deleted_ids_file (txt, ...etc...) ;

// load only the records from the incremental_data qvd that

// are not in the deleted id's file in a temporary table

T2:

LOAD * FROM incremental_data.qvd (qvd)

WHERE NOT EXISTS(ID);

// store the records from the remaining ID's

STORE T2 into incremental_data.qvd (qvd);

// drop tables that are no longer needed

DROP TABLES T1, T2;

// load the updated and new records from the nightly 7 day file

Final:

LOAD * FROM sevendaysupdates.qvd (qvd);

// load only the records from the incremental_data qvd that are

// not already in the nightly 7 day file

concatenate(Final)

LOAD * FROM incremental_data.qvd (qvd)

WHERE NOT EXISTS(ID);

// store the complete set of records in the incremental qvd

STORE T2 into incremental_data.qvd (qvd);


talk is cheap, supply exceeds demand
Community Browser