Skip to main content
Announcements
Qlik Connect 2024! Seize endless possibilities! LEARN MORE
cancel
Showing results for 
Search instead for 
Did you mean: 
Not applicable

Incremental load

Hi

i have query related modified date and last execution time in incremental load.

if modified date and primary key is not there how to implement incremental

6 Replies
arsal_90
Creator III
Creator III

if modified date is not here then in my idea you should have to make a date flag like Today() as Modified Date  and in historical data you also have to tag a flag with today()-1 as Modified date

Not applicable
Author

then what about the last execution time?

stevedark
Partner Ambassador/MVP
Partner Ambassador/MVP

Hi Venu,

If you don't have any kind of last modified date in your source data then an incremental load is going to be tricky to do.  Is it possible to get a date stamp added to the source data?  If you can not modify the data structure directly to add a last updated date you could potentially add a trigger to write one?  If the trigger wrote a record to an audit trail table giving the ID of the record and the time modified that could be used (along with an INNER JOIN) to give just the records you want.  If you don't have a unique ID on the record either then you will need to record all of the fields that are required to uniquely identify that record - all of these can then be used in the INNER JOIN to the trigger generated table.

If you do need multiple values to uniquely identify the record then you will need to concatenate those into a single field when loading the data into QlikView - as the WHERE EXISTS (a critical component of most incremental loads) statement requires this single unique field.

Note, that for an incremental to be worth doing all loads from QVD should be optimised - otherwise there is not much point to making the load incremental.

I have blogged on both Incremental Loads and Optimised Loads in the past.  I shall post a link to this thread.

Regards,

Steve

stevedark
Partner Ambassador/MVP
Partner Ambassador/MVP

The blog posting on Incremental Loads that I referred to above can be found here:

http://www.quickintelligence.co.uk/qlikview-incremental-load/

Steve

Not applicable
Author

can you please share link regarding QVD and incremental load blog.

stevedark
Partner Ambassador/MVP
Partner Ambassador/MVP

Hi Venu,

The link is presently in the moderation queue, and should appear at some point soon.  In the mean time if you Google "QlikView Incremental Load" you should find some information - including my Quick Intelligence blog post.

Steve