Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi All,
Can anyone please tell me after incremental load how do we make sure data loaded is not duplicate ??
Just wanted to know what kind of filters we put at our end to make sure redundant data is not loaded in incremental load?
Thank You
Neha Agrawal
Hi all,
can anyone please answer this question....!!
Thank You
Neha Agrawal
You usually use the primary key for that.
You load the incremental extract and then concatenate the data from previous extractions whose primary key has not been previously loaded in the incremental part.
Neha,
you need to provide what incremental technique you are using and if you have a unique key or if this is the combination between different fields, as Daniel already stated normally this is done with a primary key, though I have seen cases where the primary key is only unique for one specific date, so this is determined by your setup