topic Re: [resolved] Create SCD without using specific SCD component in Talend Studio

[resolved] Create SCD without using specific SCD component

Anonymous — Wed, 11 Jun 2014 08:12:38 GMT

Hi,
Im almost sure that I saw this solved before in some blog/forum.. but currently Im not able to find it any more.
If I remember well it was using a tMap component.
I need to to make it in this way because my DW is on Redshift and it doesn't have any specific component for it.
Any help/link is apprenticed!
Thanks!
Oscar

Re: [resolved] Create SCD without using specific SCD component

janhess — Wed, 11 Jun 2014 09:25:40 GMT

Can you be a bit more specific with your requirements.

Re: [resolved] Create SCD without using specific SCD component

Anonymous — Wed, 11 Jun 2014 09:36:51 GMT

Hi Janhess,
thanks for your answer! will try to explain, but Im looking for a generic solution that later I can apply to different scenarios.
I have a customer SCD dimension.
I need to implement the logic to insert and update depending of the new existent values in the staging table comparing it agains the dim table.
Some columns are Type 1
and others are Type 2
I just need to see some example that I can adapt later to my particular cases
I think i can use a tMap component in order achieve it, but it will be great if I can find some kind of guide that I can use to achieve this.
Thanks!
Oscar

Re: [resolved] Create SCD without using specific SCD component

janhess — Wed, 11 Jun 2014 09:59:14 GMT

Is this the sort of thing you're looking for?
https://community.talend.com/t5/Design-and-Development/update-insert-AND-delete-old-rows/td-p/100314

Re: [resolved] Create SCD without using specific SCD component

Anonymous — Thu, 12 Jun 2014 07:39:27 GMT

Hi Janhess,
thanks for you link, at the end I based my solution in this video http://www.youtube.com/watch?v=dH567IFszu4
I think its closer to what Im looking for.
Now my solution works partially good, it works well when I have only SCD1 cases, and also when I have only cases for SCD2, but the tMap seems to not be able to finish when the SCD1 and SCD2 cases needs to be processed in the same run. Also not sure if this related with redshift that may not allow to perform updates from different components.
I can provide the transformation by email if necessary, but I dont know how to attach here in the forum.

Thanks!
Oscar

Re: [resolved] Create SCD without using specific SCD component

Anonymous — Wed, 09 Jul 2014 10:12:25 GMT

Hi,
I just added a printscreean where you can see how my solution looks like.
In order to avoid Redshift locks in the updates of Type 1 and Type 2, I splited the job in two parts.
(1) In the first one I do new Inserts and Updated Type1
(2) In the second part Im making Inserts and Updates Type 2
Cheers,
Oscar

Re: [resolved] Create SCD without using specific SCD component

Anonymous — Wed, 09 Jul 2014 10:21:23 GMT

hi all,
What about using SCD Talend component ?
https://help.talend.com/search/all?query=tMysqlSCD&content-lang=en
exist for other DB (Oracle,Postgres,etc)
hope it help
regards
laurent
PS : Forget !!! don't have read title well (without scd componenet) ...

Re: [resolved] Create SCD without using specific SCD component

Anonymous — Wed, 09 Jul 2014 10:24:32 GMT

Thanks Laurent for your suggestion, that will be great, but SCD component still don't exist for Redshift, which is what Im using now.
Cheers,
Oscar

Re: [resolved] Create SCD without using specific SCD component

Anonymous — Wed, 09 Jul 2014 10:55:10 GMT

sorry so ... ask R&D to add it !!
you can really ask it with talend tracker : https://jira.talendforge.org/secure/

Re: [resolved] Create SCD without using specific SCD component

_AnonymousUser — Tue, 05 Aug 2014 18:56:19 GMT

i used the postgres scd for redshift and it seem to work but the performance is terrible! would you mind sharing your target load rows/s with your "custom" talend type 2 scd for talend? Would appreciate if you can provide some screenshots oscar!

Re: [resolved] Create SCD without using specific SCD component

Anonymous — Tue, 05 Jan 2016 08:02:24 GMT

Hi,
How to do SCD without SCD component??
Please share a screen shot or mapping
Thanks in advance
Regards,
Ramkumar
OJT-Walmart