Deadlock during multiple Talend job execution

_AnonymousUser — Fri, 20 May 2016 14:51:31 GMT

I have talend jobs configured through Talend Admin Console (TAC) in multiple instances through context groups like UK, SA etc. For these instances I have set the file triggers (like source file trigger, confirmation file trigger etc). For Eg, UK instance is dependent on both UK source file and a confirmation file from a dependent process say X.
But these 2 instances can run in parallel (since the UK and SA source files can come in parallel). When these instances am trying to insert/update into the same SQL server 2012 database table at the same time, one of the instance is failing with the deadlock error as below:
“java.sql.SQLException:Transaction (Process ID 69) was deadlocked on lock | communication buffer resources with another process and has been chosen as the deadlock victim. Rerun the transaction.” .
Hence I need to manually run the failed instance again in such cases.
Hence I am looking out for an option on TAC side itself if TAC can handle such type of parallel runs/deadlocks and wait for the other process/instance to complete successfully before proceeding with the current process/instance (so that I do not get this deadlock error and hence avoid this manual re-run).
Also, I need to know if TAC has a “RETRY” job option – which basically re-runs the failed jobs automatically “N” number of times (where “N” can be configured) – This property is present in few other schedulers and hence need to check if TAC has this kind of property.
Any inputs which will help us to resolve this issue is appreciated.
Thanks in advance.

Re: Deadlock during multiple Talend job execution

Anonymous — Fri, 20 May 2016 17:37:58 GMT

You can commit your SQL output for each and every row.(using Advanced Settings).That would prevent tables to wait in loop. But this might degrade your performance!

topic Re: Deadlock during multiple Talend job execution in Talend Studio

Deadlock during multiple Talend job execution

Re: Deadlock during multiple Talend job execution