topic Re: Slowing Changing Dimensions in Talend Studio

Slowing Changing Dimensions

DaveHarms — Sat, 16 Nov 2024 04:41:12 GMT

I am in the process of changing over to use slowing changing dimension in Snowflake table using Talend. I am doing a type 2.

This requires me load our existing data slowing changing data into the Snowflake database. I get the data loaded correctly, but when I run the Talend job, when it writes a new record it starts at 1 instead of the next number in the file. The old record is deactivated correctly and a new one written, but the version number is 1.

An example: last version number in the file 98401. The next record should be 98402, but instead starts at 1.

Re: Slowing Changing Dimensions

manodwhb — Fri, 13 Sep 2019 07:20:04 GMT

@DaveHarms ,can you share your job design , I believe you might not appending with max value into file or may be you have not do increment properly.

Re: Slowing Changing Dimensions

DaveHarms — Fri, 13 Sep 2019 16:12:10 GMT

Attached is a section the Talend HTML document. Sorry I could not attach the whole HTML document, but it contained passwords.

Not sure where append with High value would specified.

Hope this what you were looking for.

Thanks

Talend tjdbcxx.docx

Re: Slowing Changing Dimensions

tnewbie — Fri, 13 Sep 2019 17:29:05 GMT

Dave,

From the document you shared, it looks like you do not have a key defined, can you check(tick mark) the key column and select max number +1 option in the SCD editor, it might solve your problem. Pls. refer to the attachment

scd.png

Re: Slowing Changing Dimensions

DaveHarms — Mon, 16 Sep 2019 18:48:15 GMT

Thanks for the reply.

We are using the JDBCSCDELT component, the TABLE max +1 does not appear to be an option. There does appear to have a snowflake SCD component.

Any other suggestions.