topic Re: [resolved] Parallel execution with iterate (Tparallelize with Tflowtoiterate) in Talend Studio

[resolved] Parallel execution with iterate (Tparallelize with Tflowtoiterate)

Anonymous — Mon, 06 Aug 2012 18:54:18 GMT

I am trying to execute a bunch of subjobs in parallel, but context values need to be passed to the subjobs via a flowtoiterate component.
However, tflowtoiterate does not allow an 'iterate' to be linked to the tparallelize component. I can connect the tflowtoiterate directly to the subjobs however then the jobs execute in parallel and I am not able to use the Synchronize option of the tparallelize component. Can anyone suggest a way to go about this requirement. (Parallel execution for an iterative list of contexts passed from a tflowtoiterate component)

<a href=" http://tinypic.com?ref=54i00p" target="_blank"><img src=" http://i48.tinypic.com/54i00p.jpg" border="0" alt="Image and video hosting by TinyPic"></a>
also, on a side note..the contexts in the tflowtoiterate do not get the values I have assigned from the previous component. I don't know why that is.

Re: [resolved] Parallel execution with iterate (Tparallelize with Tflowtoiterate)

Anonymous — Mon, 06 Aug 2012 18:56:57 GMT

Don't know why the image wont upload. Hopefully people are able to see from the below.
<a href="http://tinypic.com?ref=54i00p" target="_blank"><img src="http://i48.tinypic.com/54i00p.jpg" border="0" alt="Image and video hosting by TinyPic"></a>
http://tinypic.com/r/54i00p/6

Re: [resolved] Parallel execution with iterate (Tparallelize with Tflowtoiterate)

alevy — Tue, 07 Aug 2012 08:21:46 GMT

Connect tFlowToIterate to tJava as a place-holder then OnComponentOK to tParallelize.

Re: [resolved] Parallel execution with iterate (Tparallelize with Tflowtoiterate)

Anonymous — Tue, 07 Aug 2012 19:20:28 GMT

Thanks Alevy-that works.
However, I am still trying to understand why the tflowtoiterate does not assign values to the contexts from the table input. Is there something else that I need to do for that?

http://i48.tinypic.com/3vvi8.jpg

Re: [resolved] Parallel execution with iterate (Tparallelize with Tflowtoiterate)

alevy — Wed, 08 Aug 2012 00:44:33 GMT

tFlowToIterate places values on the globalMap only i.e. with your config, you are creating variables that can be retrieved using syntax like (Integer)globalMap.get("context.JIVE_MULTI_TENANT_CONTROL_TenantID").
If you want to store them in the context instead, you will have to do that manually in the tJava or tJavaRow (e.g. context.JIVE_MULTI_TENANT_CONTROL_TenantID = row3.TENANTID

Re: [resolved] Parallel execution with iterate (Tparallelize with Tflowtoiterate)

Anonymous — Wed, 08 Aug 2012 18:58:50 GMT

Yes I was assigning values to the contexts using the tjavarow component. But your explanation clarifies the way the tflowtoiterate is supposed to work.
Thanks!

Re: [resolved] Parallel execution with iterate (Tparallelize with Tflowtoiterate)

alevy — Fri, 10 Aug 2012 01:07:35 GMT

There's a new Jira regarding this scenarion ( tParalellize with iteration, doesn't synchronize correctly) that I thought should be brought to your attention. Of course, the timing is suspiciously close so you might have raised the support ticket that led to the Jira in the first place...