Skip to main content
Announcements
Qlik Introduces a New Era of Visualization! READ ALL ABOUT IT
cancel
Showing results for 
Search instead for 
Did you mean: 
XaviMarti
Contributor II
Contributor II

Ordering Paralelized subjobs

Hi when paralelizing jobs is there a way of setting some "grouping".

Let's say some of them has tu run first and then the others. Otherwise the server gets stuck!

Thanks in advance

Labels (3)
2 Replies
Anonymous
Not applicable

Hello @Jose Javier Marti Camarasa​ ,

When a Job contains several subJobs, you might want to execute some of the subJobs in parallel and then synchronize the executions of the other subJobs at the end of the parallel executions.

To do this, you can simply use tParallelize to orchestrate all of the subJobs to be executed

Please refer to the below article

https://help.talend.com/r/en-US/7.3/studio-user-guide/orchestrating-parallel-executions-of-subjobs

 

Best regards

Aiming

 

Afoster1677824530
Contributor III
Contributor III

Hello,

I would like to suggest you here.

Yes, when parallelizing jobs, it is possible to set grouping to ensure that certain jobs run before others. This is often referred to as job scheduling or task scheduling.

One common approach to job scheduling is to use a job queue. A job queue is a data structure that holds jobs in the order they were received. You can define a priority or grouping scheme that determines which jobs get added to the front of the queue and which jobs get added to the back. When a worker is available, it can remove the next job from the front of the queue and process it. This ensures that jobs are processed in the correct order.

Many job scheduling frameworks or libraries support job queues and grouping. For example, the Python library Celery allows you to define task priorities and dependencies. Similarly, the job scheduler Slurm allows you to define job dependencies and submit jobs to a queue.

In addition to job queues, there are other approaches to job scheduling, such as dependency graphs or workflow engines. These approaches can provide more fine-grained control over job scheduling, but may be more complex to set up and manage.

I hope this helps! for more update must visit www.portal4me.com