Running multiple interdependent jobs one after othe

[Anonymous]

Hi,

I have a case where a Job has to be broken down to two sub jobs as I have to take the output of one job, do some modification (in the output table) and use it as an input to the next job. So, how to do it ? If i try to do something like in the screenshot below then it runs both the jobs simultaneously which I don't want.

[Anonymous]

Right click on TEST1, select Trigger and select "On Subjob OK". Drag this link to the TEMP_STAGING _TABLE component. Done.

[Anonymous]

No, I think you didn't picked my question properly. What I want is that on running the job only first part should run, i.e from TEST to temp_staging_stable, then I have to do some modifications in the temp_staging_stable, then when I run it gain, now the second part of job should run ,i.e from temp_staging_stable to Employee.

Any suggestions how to achieve this?

Regards

Pankaj

[Anonymous]

Check for a controlling state, i.e. either a variable, or some data, to identify that the first step is complete and you have done your modifications in the temp_staging_table, then depending on that controlling state, you either run the first subjob or the second subjob. 

What you will do is as follows:

tJava -> Run If -> (Your First Subjob)

          -> Run If -> (Your Second Subjob)

Yes, they are linked to the same tJava which should not contain any Java code in its body.  It is just a stub to allow you to run to If triggers from 1 point.  In each if trigger, you are checking for the correct controlling state.  You can use context variables, or actually query the DB for some of the modifications you did, before you reach the tJava to decide what is the controlling state for the If condition.

[Anonymous]

Its a bit difficult for me to design and also I want to avoid any Java coding as my client doesn't have any coding skills.

Isn't it possible that I select my firstjob, execute it, then do some modifications in the resultant table, then select my second Job and execute it!

[Anonymous]

@iburtally was not talking about using any Java. The tJava is just used as a dummy component to drive the conditional RunIfs from. 

Why are you designing this as one job? If you have to perform some manual steps between the two processes, why not just write two jobs? Don't get me wrong, it is easily achieved by Talend, but there is no point struggling for ages when two separate jobs will do.

[Anonymous]

Well that had to be the last resort, i.e. to have two separate jobs for this task as there is a manual intervention involved between tow sub jobs. All I was thinking of a way to possible 'select' one the two subjobs and execute and then repeat it for the next. But that doesn't seems to be the case so I shall go with the option of having two jobs for this.

Thanks anyways. 

[Anonymous]

What @iburtally described would be fine for this. You can also use context variables to do this. You can use the state of your data in your db to do this. You can use the presence of data in your db to do this. You can do this in SO many ways. You need to give more information about what you have to play with before anyone can be more specific I'm afraid. 

[Anonymous]

Hi @iburtally

Is it possible that at the end of step1 or first subjob I can see the result in a GUI(i.e all the columns) and can update some value of a column which updates my Database table accordingly and then I can use it as an Input to my second Job?

The purpose is that my client wants some manual approval process at the end of first job i.e. where we have applied some rule and he can see the changes and put a tick mark against the columns which he actually wants for the main table.

[Anonymous]

Hi,

We have a product called Talend Data Stewardship for this which provides GUI to curate the data.  

If you use Talend Data Stewardship, you send your data to the Data Stewardship, then you can login into the UI for Data Stewardship, curate and fix your data, then your second job read the curated data from Data Stewardship and do further processing.

If you want to build your own UI, then you can do that too.  But Talend ETL does not provide a UI.  It is supposed to be batch running without a UI.