parallel execution of jobs or components

[Anonymous]

I?ve a job which queries a lot of different slow databases and creates a summary report at the end. From the optimization perspective it would make sense to parallelize some of the queries.
Which options provides TOS to parallelize jobs or components?
I?ve seen the ?Multi thread execution? in Job Settings. But to my understanding has this option only an effect on lookup loading. The documentation is a little bit light on parallel computation. The only reference I found was in ?Defining the Start component? and from the comments there it does not look like that I can parallelize the waiting time for the databases.
I?m not sure if it would work if I encapsulate the different queries in unlinked sub jobs. The biggest issue I would see with this approach is the synchronisation before the data consolidation because I have to wait until the last job finishes. In addition I?m not sure the system really starts the unlinked jobs in parallel. Did someone already test something like that?

[Anonymous]

hi, i am using TOS for MDM v5.0.2 community version. bt I can not find tParallelize component and also there is no information about the component available in the user manual guide. please let me know if the component present in community version or where i can find a suitable example for that.
thanks, @mit

[_AnonymousUser]

It seems to me that Synchronization and Parallelism can be achieved in a Parent/Child relationship where the Parent job calls the child to do parallelism and when all parallel subjobs in the Child are done (meaning Synchronization is achieved) the Parent continues.

[Anonymous]

hi, i am using TOS for MDM v5.0.2 community version. bt I can not find tParallelize component and also there is no information about the component available in the user manual guide. please let me know if the component present in community version or where i can find a suitable example for that.
thanks, @mit

tParallelize is only available in Talend enterprise subscription product, you can read its user manual in Talend Help Center.
https://help.talend.com/search/all?query=tParallelize&content-lang=en
Shong

[Anonymous]

It seems to me that Synchronization and Parallelism can be achieved in a Parent/Child relationship where the Parent job calls the child to do parallelism and when all parallel subjobs in the Child are done (meaning Synchronization is achieved) the Parent continues.

Yes, put all the subjobs which you want to run parallel in the child job, and open the job settings, then click the Extra tab, check the 'Muti thread execution' option to make all the subjobs without any connector between them to run parallel.
Shong

[_AnonymousUser]

Talend gives us facility to execute your sub job in parallel. Its something similar like multi threading in Java.
Here is nice tutorials for parallel execution of jobs in Talend Open Studio