Multithread, iterate and tFlowIterate

[aagustoni]

Hello

I am using Talend TOS (V7), I don't have access to Enterprise version, unluckily

I have a job with the following design (see the attached screenshot)

First a query that select among the other info, n couples colorID/sizeID

Then a tFlowToIterate to make colorID and sizeID available in an iteration

On component OK the colorID/sizeID are used as filters to retrieve the data from an Oracle Table

The job works fine, but when the number of colorId/SizeId are many, it is quite slow to complete.

I have tried enabling iterate parallel parameters and things run much better (the execution time halves, of course).

The problem is that reading the documentation I have found that globalMap is not thread safe so I feel there are risks in making my design multithread (a mix up of colorIds or sizeIds during the executions would cause serious problems)

Is there a way to modify my design in order to make it thread safe?

Thanks for your help

Alessandro

Basicall

[fdenis · Accepted solution]

yes, you may iterate to trunjob using context value in place of global.
with that configuration thread have there own variables.

be careful tjava is not explicite so we do not know what is the objectif of your job. not easy for maintenance.

[aagustoni]

The tJava does some simple log, no biggies

So if I understand well I

- keep the iterate with parallel execution enabled

- use the globalMap values JUST to populate subjob context Parameters

Should I check the "use an independent process to run subjob" too?

Many thanks for your help

Alessandro

[fdenis]

please to help.
validate the solution on your side and check as solved.
By

[aagustoni]

Hello

I will for sure, as soon as I tested...

Just a detail : Should I check the "use an independent process to run subjob" too or the check in the iterate flow is enough?

[fdenis]

yes check it then you will have independent java thread.