Need help on increamental load without date and timestamp fields

[chinnuchinni]

Hello,

I have a few tables, which do not have the date/timestamp field to show when was the row modified. the tables are connected through other unique fields. What are the recommended ways to do incremental loads. If the unique fields are strings, then in my opinion, the incremental load will be very slow if we are searching for every new key in the old records to check whether its a new records or old one.

note: need to perform insert,update ,delete operations must(bcz my data should be update for every 10 minutes)

What are the recommended ways to do incremental load in this scenario

Thanks

Prvn

[techvarun]

In this case you need a sequential ID
Incremental Load without Date field

Thanks,

Varun

[Gysbert_Wassenaar]

Inserts and deletes are no problem if you have a primary key in your table. But if you don't have a field that indicates that a record has been updated then you need to calculated hashes over all the fields of the record except the primary key and then compare all the hashes per primary key to identify the changed records. See this blog post for an example: http://www.qlikfix.com/2014/03/11/hash-functions-collisions/

You should really urge the database administrator of the source database to add a modification datetime field to the table that is filled with a trigger on insert and update.