topic Re: tDBOutput "Action on data" Delete is extremely slow in Talend in Talend Studio

tDBOutput "Action on data" Delete is extremely slow in Talend

Tasfiahm — Fri, 15 Nov 2024 22:15:00 GMT

In my job I am reading a .csv file > passing the value in t-map > doing an Inner Join with the look up table> trying to perform tDBOutput "Action on data" Delete in the target table

The delete is extremely slow even though in am using Talend Data management Platform 7.2.1.20210302_1138 enterprise edition.

I can do a different design approach by loading the input data into a Temporary table and then perform the delete in SQL row component.

This is a very basic functionality. So is it a Talend issue or I am doing something wrong in the configuration of the component?

"I believe there is always something new to learn.

Re: tDBOutput "Action on data" Delete is extremely slow in Talend

jlolling — Thu, 22 Dec 2022 19:49:39 GMT

This is NOT an Talend issue. I am pretty sure the performance issue is caused by the attempt to delete data in a table without using an index on the table.

I would try to run the delete for testing in an SQL tool and check the execution plan.

If this works fine and the table has the right index applied to you can continue to improve the Talend job. E.g. you can check if you use an auto-committed connection or you miss to setup batch mode in the tDBOutput.

You should use separate tDBCommit component and check if you have enabled the batch mode in the Advanced settings of the tDBOutput.

BUT first take care about the table index fits to your key to delete!

Re: tDBOutput "Action on data" Delete is extremely slow in Talend

Tasfiahm — Thu, 22 Dec 2022 20:46:20 GMT

Hi jlolling,

After creating the index on the DB table it solve the performance issue.

Thanks a lot for helping me to solve the issue.

😁 😁 😁