tStatCatcher statistics causing nullpointerexception within parallel execution (it uses HashMap within Threads)

ekilby — Thu, 03 Jan 2019 10:31:39 GMT

I'm finding that some of my talend DI jobs are failing with a nullpointerexception at random times. Looking a the line of code failing, it is the tStatCatcher addMessage method being called to record the duration of the job, like the following example:

tStatCatcher_1.addMessage("end","tFileTouch_4", end_Hash.get("tFileTouch_4")-start_Hash.get("tFileTouch_4"));

This only happens on components that happen to be within a parallel executed iteration flow (e.g. I'm running four at a time )

e.g.

I capture the exception as part of the tStatCatcher framework

This is the exception message:

java.lang.RuntimeException:Child job running failed.
java.lang.NullPointerException
at reporting_etl.etl_con_job_0_1.etl_con_job$1tFileTouch_4Thread.run(etl_con_job.java:6737)
at routines.system.ThreadPoolWorker.runIt(TalendThreadPool.java:159)
at routines.system.ThreadPoolWorker.runWork(TalendThreadPool.java:150)
at routines.system.ThreadPoolWorker.access$0(TalendThreadPool.java:145)
at routines.system.ThreadPoolWorker$1.run(TalendThreadPool.java:122)
at java.lang.Thread.run(Thread.java:745)

line 6737 is that addMessage method call above

I've noticed that Talend use a HashMap for putting and getting tStatCatcher messages, which is not thread safe and as this is in a parallel execution (threaded) then I'm assuming that it's a threading issue with the Talend Generated code, this combined with the fact that it is randomly happening leads me to believe that they should be using a Concurrent HashMap.

Does anyone have any advice on what I can do here, shall I raise a bug, or is there a work around for this?

Kind regards

Re: tStatCatcher statistics causing nullpointerexception within parallel execution (it uses HashMap within Threads)

Anonymous — Fri, 04 Jan 2019 08:12:54 GMT

Hello,

Could you please clarify in which Talend version/edition you are?

Best regards

Sabrina

Re: tStatCatcher statistics causing nullpointerexception within parallel execution (it uses HashMap within Threads)

ekilby — Fri, 04 Jan 2019 08:58:41 GMT

Hi, I'm using

Talend Open Studio for Data Integration

Version: 6.4.1

topic tStatCatcher statistics causing nullpointerexception within parallel execution (it uses HashMap within Threads) in Talend Studio

tStatCatcher statistics causing nullpointerexception within parallel execution (it uses HashMap within Threads)

Re: tStatCatcher statistics causing nullpointerexception within parallel execution (it uses HashMap within Threads)

Re: tStatCatcher statistics causing nullpointerexception within parallel execution (it uses HashMap within Threads)