How to implement type 2 scd through tmap in spark job

RP1629548713 — Fri, 15 Nov 2024 23:16:26 GMT

Hi ,

The requirement is to develop type 2 scd using tmap in spark batch job using talend

I have 2 tmaps : tmap 1 for catching the inserts and updates. From tmap 1 there is one flow pointing to inserts in which i have checked the catch inner join reject as true. And the output2 to another tmap where i am trying to catch the updates and unchanged records.

But none of the record is being flowed as expected.

Please suggest

Re: How to implement type 2 scd through tmap in spark job

Anonymous — Tue, 01 Mar 2022 08:27:41 GMT

Hello,

In BD framework, there are no SCD specific components in BD Batch framework, where as the same functionality already exists in standard framework i.e tDBSCD which has options to provide the type fields for attributes & version for attributes.

Best regards

Sabrina

topic How to implement type 2 scd through tmap in spark job in Talend Studio

How to implement type 2 scd through tmap in spark job

Re: How to implement type 2 scd through tmap in spark job