topic Re: how to update bulk records in redshift? in Talend Studio

how to update bulk records in redshift?

k526 — Sat, 16 Nov 2024 09:39:46 GMT

HI ,

I need to load daily lakhs of records in redshift.so i need to insert or update the data.by using tredshiftbulkoutputexec we can do insert operation only.by if i used tredshiftoutput for 1 lakh records it takes more than 6 hours.

is there any way to update ?

can any one help me ?

Thank you

Re: how to update bulk records in redshift?

vapukov — Thu, 08 Jun 2017 06:57:05 GMT

lakhs of records

When You use a local terms, ожидайте неожиданного ответа

but in any case:
- bulk insert into stage table

- INSERT FROM SELECT WHERE NOT EXISTS

- UPDATE WHERE EXISTS

it also will be hard operation, but much faster than remote INSERT/UPDATE

Re: how to update bulk records in redshift?

majum2p — Fri, 01 Sep 2017 11:49:16 GMT

I too have similar requirement except i have quite a few transformation logic. i am using tRedshiftInput , tMap and tRedshiftOutput for insert and tRedshiftRow for update. looking for any other patterns for performance optimization.
P.S. creating stage table is not a good solution for me as i have hundreds of tables to update/insert.

Re: how to update bulk records in redshift?

Anonymous — Thu, 09 Jul 2020 09:07:56 GMT

How we can use tredshiftrow if the insert/update has to be made from postgres source data.

for example tpostgresinput --> tmap -->tredhsifptoutput?

Re: how to update bulk records in redshift?

Anonymous — Thu, 09 Jul 2020 09:07:56 GMT

How we can use tredshiftrow if the insert/update has to be made from postgres source data.

for example tpostgresinput --> tmap -->tredhsifptoutput?

Re: how to update bulk records in redshift?

Anonymous — Thu, 09 Jul 2020 14:00:30 GMT

Did we have solution for the below.

Re: how to update bulk records in redshift?

Anonymous — Thu, 09 Jul 2020 14:00:30 GMT

Did we have solution for the below.