Help needed||SCD 2 implementation in Talend

[Anonymous]

Hi All,

I am trying to implement SCD 2  using tJDBCSCDELT component to load the data to Snowflake. I am able to load the data successfully but i am facing a challenge. Below is the scenario.

when i loaded the data for the first two days i am successful. Below is the Screenshot

But on the day 3 when i loaded the same day 2 file, instead of filtering out the day 2 records which are already present in the target, it is appending the day 2 data as new records.Below is the screenshot for the same

configuration that i used in the tJDBCSCDELT  component. Below is the screenshot.

Please give your valuable suggestions and thoughts in resolving my issue.

Thanks in advance!

[Anonymous]

@siva_chonga can you check if the source table name and the scd table name are not the same for the daily process? Also remember that the use case to use this approach / technique is being able to identify changes not only in row level but also in attribute level for a whole dataset between one snapshot and another snapshot.

Step 1: Load new snapshot into the scd table

Step 2: Process source table and scd table using tJDBCSCDELT

Step 3: Update the source table with the results once you know the changes that you are looking for

SCD management methodology

https://help.talend.com/reader/nyO4TB3z5AFPGsaZ8~zXUw/2H4nn1_GRV4s0tFx89lmXQ

Remember that this example is only an approach, you need to define the pattern required in based in your use case and take a decision how you want to implement it.

I hope that helps.