Redshift Incremental load

Anonymous — Sat, 16 Nov 2024 05:14:08 GMT

Hello,

I'm trying for a better approach on how to do the incremental load to Redshift. Source might be Hive or S3 or any db

Without deletes, I would like to do the updates. Please give me a high level of what components and approach would be better.

Thanks

Re: Redshift Incremental load

Anonymous — Fri, 19 Jul 2019 15:00:38 GMT

Hi,

Data update is always a costly operation in any data warehouse and same is the case in Redshift also. You can use tRedshiftOutput component to perform the update operation. But if would be a good idea to design your tables in Redshift in such a way that it will always accept multiple records for same underlying customer information and you will pick only the latest record for an id (even if multiple records are present) for downstream operations (choose your indexes also accordingly).

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

topic Redshift Incremental load in Talend Studio

Redshift Incremental load

Re: Redshift Incremental load