Implementing SCD2 using TOS SCD components

[sushantk19]

Hi,

I have to implement SCD2 logic(Insert/update) in my TOS code using SCD2 components. My Target DB is Amazon Redshift. Can you please let me know which TOS component needs to be used here ?Will tMysqlSCD or tOracleSCD work ?

[sushantk19]

All inputs on above query. How do I implement SCD2 in Redshift Database?

[manodwhb]

@sushantk19 , I do not think that we have Redshift SCD component in Talend, you can not use MysqlSCD or tOracleSCD for Redshift Database.

[Prakhar1]

Hi, 

try to use JDBC component for that.

You can configure the jdbc url for your Redshift connection.

Don't forget to give Kudos

[sushantk19]

Presently, I am trying without SCD component. But facing the below issue.Its not even Inserting data in my Target table. Please see the screenshots attached. I already have 2 country ids in my target. Source  has 253 unique country ids, so as per the Insert update logic it should insert 251 and update 2 records.

Job_Design_SCD2.png
promotion_order.csv
tmap_output.png

[sushantk19]

I tried both of them :

Please see the below error:
Exception in component tDBSCD_2 (job_CORE_000_Country_test)
java.sql.SQLRecoverableException: IO Error: Got minus one from a read call

with tMySQLSCD getting communication Link error although all the DB connections are correct.

[sushantk19]

what should i give in JDBC URL? I have very limited knowledge on Java. If possible, can you share your job design?

[Prakhar1]

I have never created job like this so can't share the job design now.

The JDBC URL will look like this : 

jdbc:redshift://<endpoint>:<port>/<database>

You can also go through this link to create connection to the JDBCSCD component

https://docs.aws.amazon.com/redshift/latest/mgmt/configure-jdbc-connection.html

[sushantk19]

@Prakhar @manodwhb thanks!. I used the postgresqlSCD component and worked perfectly fine for my requirement! Is there any issue with this component for implementing SCD2?

[manodwhb · Accepted solution]

@sushantk19 , you can implement scd2 with postgresqlSCD component.