Hi,
We have a need to implement SCD methodology and have the output write to Redshift using TOSBD.
We originally tried to use the
tDBSCD component, but found the performance to be very poor (~.15 rows/sec).
Then switched to using a custom tMap with four outputs (insert new, type 1 update existing, type 2 insert new, and type 2 insert existing), just like is shown here:
https://www.youtube.com/watch?v=dH567IFszu4. Performance was better there, but still not great (~9 rows/sec).
Recently heard that we might have better luck with
tJDBCSCDELT component, instead, but haven't taken a look at that one yet.
Rather than continuing to try different methods like this, curious if someone will share what components/design they're using to write to Redshift and what kind of performance you see with that method.
