Process deadlock for hours for no apparent reason

[jhuebler]

Hi,

We use Talend Studio 7.3 and our processes are executed on premise in Windows Machines.

Since months we observe a strange behaviour from time to time:

The process starts and does it's thing for quite some time... then without any errors the progress just stops. The job within TAC is still running, but it doesn't make any progress.

This can stay like this for sometimes an hour, sometimes even three hours. Out of a sudden the progress continues.

The resuming logfile for these dead locks really look like

"2023-03-16 02:09:10.586","HWwZOK","20230316000500_4kimY","TsDV0T","CHECKPOINT","CONNECTION:SUBJOB_OK:tJava_2:OnSubjobOk","","MAIN_ETL","ProcessXYZ","PRD","0.1","","","","","",""

"2023-03-16 05:11:29.64","HWwZOK","20230316000500_4kimY","TsDV0T","CHECKPOINT","CONNECTION:SUBJOB_OK:tDB2Input_2:OnSubjobOk","","MAIN_ETL","ProcessXYZ","PRD","0.1","","","","","",""

The same process on the same server can run effortless for weeks and suddenly the deadlock appears.

On the server itself we observed suddenly heavy writing operations within the time range the deadlock appears. As if the server is too busy doing something so that the tMap operation within our process just gets stuck in the traffic jam. By the way: I is always a tMapping where the deadlock appears.

Is it possible that this deadlock is created by java itself? How can we avoid this?

Did anyone ever observed something like this?

This is super strange and neither our database admins nor the server admins found any issues on their side.

Thanks a lot

Jan

[Anonymous]

Hi

I think the best way for you to find a solution to this is to contact support. They will be able to hold a Zoom meeting with you and take a look at your Job and environment to see how best to resolve this.

Regards

Shong

[michael254]

Hello,

I would like to suggest you here It is possible that the deadlock is caused by Java itself, as Java applications can experience deadlocks if multiple threads are waiting for each other to release a resource.

One way to avoid deadlocks is to make sure that shared resources are accessed in a consistent order by all threads. You can also use synchronization mechanisms like locks or semaphores to ensure that only one thread can access a shared resource at a time.

In addition, you may want to investigate the heavy writing operations that occur during the deadlock, as this could be a contributing factor. Consider monitoring the server's CPU and memory usage during the deadlock to see if there are any spikes that coincide with the deadlock. Avalon Access Customer Service

It may also be helpful to review your Talend job design and ensure that it is optimized for performance. For example, you can try breaking up large data sets into smaller chunks, using caching or lookup tables to reduce database queries, and minimizing unnecessary data conversions.

If the issue persists, you may want to consider contacting Talend support for further assistance. hope so it is useful for you .