Exchanging values between Parent and child job - separate process

[Anonymous]

Hi,

I am able to pass values from parent job to child job through tRunJob Context Parm if both are running as independent process.

However, the reverse is not working - Returning values from child to parent job as independent process.

How can I achieve running parent and child job as independent process but sharing values?

As I would like to segregate the functionality between parent and child job. 

Parent job - restartability and passing each child job specific values.

Child job - generic functionality and run for the value it receives from parent job.

This would be one of the key things(running as individual process) that I am planning to project my solution.

Could you kindly help.

Thanks.

[Bluemoon]

The only way you can pass data from child to parent is using context variable.
Simple solution is to store data in context variable which is object

[Anonymous]

Thank you.

Passing value through context variable was not successful even.

Could you kindly reply on this post:

https://community.talend.com/t5/Design-and-Development/Passing-last-processed-record-details-from-ch...

And pass some reference to pass context from child to parent both running as independent process.

Would wanted to pass a collection.

Thank you.

[Anonymous]

Hi,

      I believe your intention is to run the parent and child jobs with more control and parallelism. But if you are running the parent and child jobs as independent, you will not be able to retrieve the data back from child to parent as parent will not be waiting for the completion of child job execution.

      So why don't you try tparallelize component to run multiple functionalities in parallel manner yet can have overall control in data transfer.

     If the answer has helped to resolve your query, could you please mark the topic as closed? Kudos are also welcome 🙂

Warm Regards,

Nikhil Thampi

[Anonymous]

I would be using tparallelize component for running parallel child jobs.

However, this is for restartability and checkpoint.

When the one of child jobs fails, restarting should start from the last processed id and not from 1st record.

To achieve this when the child job fails it would return the last processed id and will be saved in table, which will be used for restart.

The successful/unsuccessful execution of the child job is also saved into the table.

So, for these reasons I would like to pass the last processed id to parent job, as parent job reads the restart table.

Hope this helps.

[Anonymous]

Hi,

    If restartability and check pointing is your query, there is already an option available in Talend. Could you please create the jobs with check pointing feature as mentioned in below article.

https://help.talend.com/reader/pzEsR9WAQlQkz4tFLp6TpA/omFXPdTqrImC7U0HX5iVVQ

If the answer has helped you, could you please mark the topic as resolved? Kudos are also welcome 🙂

Warm Regards,

Nikhil Thampi

[Anonymous]

Hi Nikil,

Along with restartability and checkpoint I also wanted to run the jobs in parallel.

I also failed to mention that these jobs are being built to anonymise huge volume of data and will run from Oracle unix server and not in TAC.

Could you kindly provide some references to pass value from child to parent both running as separate process.

If you have any design that supports - checkpoint, restartability, parallelism to run as jar, please provide suggestions.

Per my design, I am planning to have  checkpoint, restartability, parallelism features in parent job by:

read the rule table to check if this is normal run or restart run, if normal run start from initial record id else read from last processed record id.

parallely execute child job for different ranges

every successful/unsuccessful execution updates the status with last processed record id.

Child job read the range of data to be anonymised, anonymises and writes to target.

It would be great if I can get reply by today.

Thanks.

[Anonymous]

Hi,

Could you provide some references please.

Thanks.

[Anonymous · Accepted solution]

Hi,

    First of all my sincere apologies for replying late as I was held up with my work related tasks. I normally do Talend community work during my break time.

   Below link has the Talend help article which specifies how to pass variable between parent to child. Could you please refer for your use case? If you are doing an independent process, parent will not wait for the response back from child. So you need to design your jobs keeping this factor in your mind.

https://help.talend.com/reader/Wi~~2Q0v6A43uBlbET9gQg/MHs41sIsaV91cFGXj4DVMQ

  If the answer has helped you, could you please mark the topic as resolved? Kudos are also welcome 🙂

Warm Regards,

Nikhil Thampi