Skip to main content
Announcements
Have questions about Qlik Connect? Join us live on April 10th, at 11 AM ET: SIGN UP NOW
cancel
Showing results for 
Search instead for 
Did you mean: 
Not applicable

Incremental Load Doubts

Hi All,

Can anyone please tell me after incremental load how do we make sure data loaded is not duplicate ??


Just wanted to know what kind of filters we put at our end to make sure redundant data is not loaded in incremental load?


Thank You

Neha Agrawal

3 Replies
Not applicable
Author

Hi all,

can anyone please answer this question....!!

Thank You

Neha Agrawal

danielrozental
Master II
Master II

You usually use the primary key for that.

You load the incremental extract and then concatenate the data from previous extractions whose primary key has not been previously loaded in the incremental part.

ramoncova06
Specialist III
Specialist III

Neha,

you need to provide what incremental technique you are using and if you have a unique key or if this is the combination between different fields, as Daniel already stated normally this is done with a primary key, though I have seen cases where the primary key is only unique for one specific date, so this is determined by your setup