topic Start a Talend shell script at a particular step in Talend Studio

Start a Talend shell script at a particular step

rtd6322 — Thu, 29 Nov 2018 13:31:17 GMT

I have a Talend job with five (5) tSystem steps. These run in batch executed from the _run.sh shell script. If tSystem_3 fails, I need to be able to re-run the script starting at the failed step. These are long running steps and so being able to start as the failed step is critical.

Is there away to restart the script at the failed step without having to disable the steps in Talend Open Studio and rebuilding the job?

Re: Start a Talend shell script at a particular step

Anonymous — Fri, 30 Nov 2018 08:09:21 GMT

Hello,

If we understand your requirement very well, it seems you are looking for talend recovery checkpoint capability.

You can set up at Job design time to allow processes to be resumed from one of the checkpoints if an error occurs.

Could you please have a look at this article about:https://community.talend.com/t5/Design-and-Development/Error-Recovery-management/ta-p/171 to see if it can meet your needs?

This feature will be available in talend subscription solution not open source.

Best regards

Sabrina

Re: Start a Talend shell script at a particular step

vapukov — Fri, 30 Nov 2018 09:41:42 GMT

subscription version support checkpoints

but generally, realize checkpoint for tSystem (or subjobs) it not a hard

all what you need:

- store fact - job started (with unique execution id)

- store success fact of each tSystem or Subjob

- store fact job finished successfully

after this, at the start, you need to check - if any job id does not have all steps and final marked as Ok and use tRunIf trigger for fire skipped component

more complicated (and also not resolved in subscription version) - what to do with data? sometimes you need rollback (clean) changes from a failed step, sometimes continue - this part more complicated (at least no single recommendation there)

Re: Start a Talend shell script at a particular step

rtd6322 — Mon, 03 Dec 2018 15:48:03 GMT

Thanks for the info... I modified it so each step is its own shell script to get around the restart.