parallelization

[Anonymous]

Hello All, I have a question

I have nearly 30 jobs , which i need to run daily, so i wanted to do parellization for those 30 jobs , which way do you think it is better. Can anyone please suggest a solution for me.

Thanks In Advance

Manish

[DataTeam1]

You can build jobs as stand alone (https://help.talend.com/reader/JdTBzKszzXoWvjpEJD3EBA/V5yP4nItw687WwHLVpxwFQ) and setup the execution plan for example in Windows scheduler or cron on server.

They can be executed at the same time if they are independent.

[Anonymous]

So, i should select all 30 jobs and from the tree view and make them into standalone and then process it as per documentation in  the link

[DataTeam1]

No, you should build separately each job as stand alone. If you have job which runs other subjob you have to right clik only on the main job in repository view and select build job.

[DataTeam1]

If you want to run many subjob in parallel in main job read this: https://community.talend.com/t5/Design-and-Development/Can-I-run-different-subjobs-in-parallel-in-a-...

[Anonymous · Accepted solution]

@ksingh 

Could you please refer the Talend help link below? It is having multiple options. Please note that some of the options are available only for subscription product.

Based on the product you are using, could you please select appropriate strategy?

https://help.talend.com/reader/MNcEDgjyM49yQ58GboyG4Q/efgfWM4DK8gYYWPhJCnTjw

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

[Anonymous]

@nthampi  the link you sent me 

https://help.talend.com/reader/MNcEDgjyM49yQ58GboyG4Q/efgfWM4DK8gYYWPhJCnTjw

I just followed same steps i need to execute 30 jobs, initially tried with 4 jobs, this worked accurately as per my requirement , thanks @nthampi 

[Anonymous]

Hi,

   Please note that you may not be able to run 30 jobs in parallel unless your job server is having sufficient compute capacity. Each subjob you are calling will consume some CPU and memory. So the wrapper job's Xms and Xmx parameters should be high enough to cater the resource requirements of subjobs.

   So tparallelize means you can do parallel processing as long as you are providing sufficient compute resources.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂