Are globalMap Variables specific to a row pass or do they interfere?

[Anonymous]

Hello, 

I need something clarified:

Are globalMap Variables specific to each single row pass (1 input flow / document) ? 

Or do they interfere with each other ?

Background to the Question:

I am expecting alot of input documents and I will need to use if conditions to control which part of my job handles which document. 

My Problem is now: It's very likely that another (if not many others) documents get processed (and arrive at the first if condition) while a document is still being processed (and still depends on variables for decisionmaking). 

If I use globalMap Variables for this, will the incoming documents / "rows" interfere with eachother's variables? 

If yes, is there anything I can use instead?

I haven't been able to find anything regarding this in the documentation so I apologize if this has been answered somewhere already.

[TRF]

Do you have parts of your job which are parallelized?

If so, take to not reuse the same variable on these parts.

Also, if you want to "secure" your global variable you may have dynamic names.

For example, suppose you have a variable called "myVar" and you have a field called id on your document.

When you create the global var, you can use the following expression to make its name unique:

row1.id + "_myVar" // instead of "myVar"

Does this help?

[Anonymous]

The Job is not parallelized, no. 

Dynamic Variables might be a solution but that would massively swamp the memory would it not? 

I'm expecting the job to run non-stop with several hundreds of incoming documents per day.

Furthermore: How would I use such a dynamic Variable in an "Execute when" connection?

Reading between the lines, there are not row pass specific flags/variables we can use?

Or can we somehow free up these dynamic variables ?

[Anonymous]

Is there maybe a variable available in Talend that identifies a specific iteration / row pass?

That would solve my Issue hopefully, the dynamic variables sadly do not (not every subjob accesses the document and so won't be able to access the dynamic variable).

I'd be really grateful for any info / help on this. 

[TRF]

Can you share your job design?

[Anonymous]

I cannot share the actual Job design, but I've attached a example with a similar Structure.

The gist is: 

First each incoming document gets analyzed and various data is extracted. 
Based on this flags are set to determine how this document needs to be processed (multiple flags can be true / 1 document can be processed in more than 1 subjob).

After that there are "Execute if" connections to the processing subjobs. 

Until recently I reset the flags inside these subjobs, but our input documents changed and now the job has to account for the possibility that 1 document needs to be processed in more than 1 way / subjob. 

This in return prevents me from resetting the flags inside the respective subJobs and am now afraid that the flags will get messed up by following documents while a respective document is still awaiting processing.