TLogCatcher_1 -> tMap -> tDBOutput performance issues

gcjp — Sat, 16 Nov 2024 00:59:06 GMT

Hello,

We are using AWS Redshift. We have a shared etl_execution_log_history table to which I log to from several jobs.

The job inserts whenever a tWarn is run.

This means we are logging around 700 rows in a 30 minute period. The tDBOutput is logging only 1 record at a time, which is causing the load to take considerably longer than expected, as Redshift does not handle single inserts very well.

We have tried using the Extended Insert option but this doesn't seem to work as I can still see single rows being inserted, even after this option has been checked on our logging.

Please would someone assist with some suggestion as to how we can avoid this single row insert issue?

Thanks

Re: TLogCatcher_1 -> tMap -> tDBOutput performance issues

Anonymous — Wed, 09 Dec 2020 04:14:55 GMT

Hello,

Could you please post your current job design screenshots here which will be helpful for us to address your issue quickly.

Best regards

Sabrina

topic Re: TLogCatcher_1 -> tMap -> tDBOutput performance issues in Talend Studio

TLogCatcher_1 -> tMap -> tDBOutput performance issues

Re: TLogCatcher_1 -> tMap -> tDBOutput performance issues