Global Map and Parallelization and Iteration

[mborlo15]

I noticed in this documentation link

that there apparently is some unreliability with global Map when parallelization and/or iterate components are used. I wanted to better understand why this is the case. I have two scenarios that use parallelization, global variables, and iterate connections. Would these uses of the features cause issues with the global map?

[gjeremy1617088143]

Hi, I think it's when you use iteration with parallel execution activated, so iteration are processed in parallel and because you set a globalVar in the iteration it can be overwritten by an other iteration in parallel.

eg: you have one iteration executed in paralllel, a first iteration with a globalVar set to 1 an other with the same globalVar set to 2 maybe 2 will be used in the first one or 1 will be used in the second one, because they are not processed one after one but in the same time.

Send me love and kudos

[mborlo15]

Ah ok, so my examples above would be unaffected because each thread would have it's own isolated instance of the child job, right?

[gjeremy1617088143]

I think Yes

[mborlo15]

Can Talend employees confirm this? It’s key for me to understand this so that the right data is sent to the right places. @Xiaodi Shi

[Anonymous]

Hello,

We will check it with our R&D team and come back to you as soon as we can.

Best regards

Sabrina

[mborlo15 · Accepted solution]

See @Richard Hall​ 's response on this matter. It is a result of globalMap being an instance of java.util.HashMap, which is not thread safe.

https://community.talend.com/s/feed/0D55b00006viq18CAA?language=en_US

for more details see

https://stackoverflow.com/questions/18542037/how-to-prove-that-hashmap-in-java-is-not-thread-safe