Scd type 2 without using component

[lmit]

Hi all,

How can I implement scd type 2 without using scd type 2 components?

For sample
Id,name,course,key
1,xxx,jjj,k1
2,yyy,KKK,k2

For first load data will be loaded as its is.for second load if data is like below
Id,name,course,key
1,ccc,jjj,
3,ddd,lll

My output should be

Id,name,course,key
1,xxx,jjj,n
2,yyy,KKK,k2
1,ccc,jjj,y

The key column for old record should change to n and new one should be y

Any help would be helpful.

Thanks in advance,
Lmit

[vboppudi]

Hi lmit,

what about "3,ddd,lll " this record in second run? Do we need to load to target or reject?

What is the key value for new records is it 'Y' or null when inserting first time.

Regards,

[lmit]

Hi Veeranjaneyulu,

Thanks for your reply

3,ddd,lll
Will insert as a new record and key will be y. It will be empty while inserting for the first time.

Thanks,
Lmit

[manodwhb · Accepted solution]

please find the below links.

http://www.deepinopensource.com/scd-2-implementations-in-talend/

http://talenddev.blogspot.in/2016/03/apply-scd-without-using-scd-component.html

[lmit]

Thanks Veeranjaneyulu and Manohar for your inputs.I achieved SCD type2 in other way

Thanks,

lmit

[Anonymous]

Hi limit,

Can you please explain how can you achieve SCD2 without using SCD compoenent.

Thanks,
Arpit

[lmit]

Hi Arpit,

I implemented using tmap with 3 flows ... 

In first flow : i have compare the columns on which columns we need to implement the scd and lookup on one unique(Primary key) cloumn 

1)if the id(lookup) had match and compare on any compared columns failes it should go to update flow(flag will change)

2)if the id(lookup) didnot match , then that record should go to insert flow(Flag will change)

3)if the id(lookup) and compare columns matched, then the record will go to ingore flow

according to my requirement i need to change the flag.

What is your requirement , do you need to generate key for update records ?

Thanks,

lmit

[Anonymous]

Hi Lmit,

Thanks for giving your response.
My requirement is:
1. if data is fresh it will load DB and generate Start date as current data and End date as null.
2. if data is already exist in DB but one attribute change then it will generate End date for previous record and insert new update record in DB with Start date as current date and End date as null.

Lmit, can you please share me screen shot of your Job and tmap that how you put expression on expression builder and what had you wrote expression for updateflow if data is already present in DB.

Hope you will understand my requirement.

Thanks,
Arpit

[manodwhb]

@arpiita, check the below link might be usefull for your case.

https://community.talend.com/t5/Design-and-Development/Flag-not-updating-based-on-condition/m-p/2682...

[lmit]

Hi arpit,

Sorry for the last reply . please find the attachment for the screenshoots , follow the names  of the screenshoots.

Hope it will be helpful for you to build the job.

Thanks,

lmit

SCD2.zip