Execute multiple flows in parallel after one component

[Anonymous]

Hi, I am trying to create a job with TOS DI 7.0.1, where I can have multiple flows run in parallel after one component say for example a tJava component. Below is a screen shot. In the first step I want to create a concatenated string which will be the parameter for tDBInput_2,tDBInput_3,tDBInput_4,tDBInput_5 and tDBInput_6. Once that is created i am doing little bit more manipulation inside tJava_2. Once that is done i want to execute tDBIput_2,tDBIput_3,tDBIput_4,tDBIput_5 and tDBIput_6 all in parallel. How can i achieve that. I do not see tParallize in pallet. Thanks in advance.

[David_Beaty]

Hi,

That component is probably missing because you might be using the Open Source version of the product (which doesnt have that functionality).

You can get around this by creating a subjob with just the 5 tDBINput flows in, then link that subjob in to your current job. Ensure on the subjob, properties enable multi-thread execution.

The only caveat with any parallel processing strategy is consider whether your Talend execution server or database server can cope with these 5 processes being run together.

[Anonymous]

Thanks for the response. Is there any way i can get around this without a sub job?

[akumar2301 · Accepted solution]

There could be many ways to implement 
 
Step 1 ) use Multi-Thread execution Option in TOS
Step 2)
option1) tWaitforFile -
 
At the end of your main subjob , create/update/delete a file which will trigger all your subjob in parralel using tWaitforFile ( go with exit loop option) . Below link will help you how it work.
 
https://help.talend.com/reader/wDRBNUuxk629sNcI0dNYaA/bnMDJWu5G0EaJl8_KxcYyA
 
Main Subjob -- > trigger a action
twaitforfile_1 --> (main) --> tlogrow --> on subjobOK --> tdbinput_1
twaitforfile_2 --> (main) --> tlogrow --> on subjobok --> tdbinput_2
 
option2) tLoop ( While condition) 
 
Create a global var VAR1 with which is true. At the end of your main subjob make VAR1 as false.
 
tLoop_1--> while VAR1  is true --> tJava ( does nothing)
|
On Subjob Ok ( when VAR1 will become false) 
[
tDbInput_1
 
Both above approach error handling of main subjob should be done properly.

Option3) Use multiple tPostJob ( if not already used and Not recommended option)
 
tPostJob_1 --> tDbinput_1
tPostJob_2 --> tDbinput_2
 
There could many other ways .
Let me know if it works.