Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Join us to spark ideas for how to put the latest capabilities into action. Register here!
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

Redshift Incremental load

Hello,

 

I'm trying for a better approach on how to do the incremental load to Redshift. Source might be Hive or S3 or any db

 

Without deletes, I would like to do the updates. Please give me a high level of what components and approach would be better.

 

 

 

Thanks

Labels (4)
1 Reply
Anonymous
Not applicable
Author

Hi,

 

     Data update is always a costly operation in any data warehouse and same is the case in Redshift also. You can use tRedshiftOutput component to perform the update operation. But if would be a good idea to design your tables in Redshift in such a way that it will always accept multiple records for same underlying customer information and you will pick only the latest record for an id (even if multiple records are present) for downstream operations (choose your indexes also accordingly).

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂