topic SCD type 2 how to track deleted source rows? in Talend Studio

SCD type 2 how to track deleted source rows?

Anonymous — Wed, 30 Jan 2019 10:09:49 GMT

Hi!

Is it possible to track deleted rows from the source with SCD 2. What I expect is that the current row in the target db will get an scd EndDate if the key was not found in the source input (csv).

Is this how it is supposed to work in Talend for SCD 2?

Any ideas or suggestions for how to cope with deleted records in source files is appreciated.

Thank you in advance.

Kind regards

Re: SCD type 2 how to track deleted source rows?

David_Beaty — Wed, 30 Jan 2019 10:26:31 GMT

Hi,

I would have thought that the CDC strategy on the source system needs to change to report all rows that have had something happen to them (inserted updated deleted). Depends if you have a proper CDC mechanism or you are just selecting rows from source with a created/update date in the range of interest.

Not really a definitive answer but how Talend will handle it is probably second to actually getting the update to Talend.

Re: SCD type 2 how to track deleted source rows?

akumar2301 — Wed, 30 Jan 2019 10:46:25 GMT

One way to identify it using tMap using Outer Join

Keep Your Table in main And your CSV file in lookup and do Outer Join based on Keys.

If you do not find the record in Lookup , populate end date and then update your Table.

Let me know if any issue.