Hi,
I am working on a requirement, where I need to find out new and updated records from a full delivery from source system to Data warehouse. I need to capture those changes and process them into SCD table.
- I have a table called 'A', where I get a full delivery. Then, I load data from table 'A' to table 'B' with some additional look ups. Table 'B' has action on data insert or update. It also has primary key, so it finds new records and updates changed record.
- I also have set up Change Data Capture on table 'B' in order to get changes and from table 'B', I load them to a SCD table.
- The problem here is that - even when there is no change in records, the records are updated in table 'B'. And I get all of them through CDC. For example - If I execute 10 records for the first time, it gets through and I get 10 records. When I execute the same 10 records again, it should not give me any records as there is no change is the data. But CDC gives me the same 10 records again.
- I have checked the trace log, with insert or update option on table, talend updates even if there is no change.
How do I avoid updating when there is no change so that it doesn't come through CDC?
