topic Re: Recovery point in Talend Studio

Recovery point

Anonymous — Mon, 03 Nov 2014 10:49:07 GMT

Hi
I have 5 jobs that are running in parallel. if anyone of the jobs fails i want to start that job only. I can set recovery point on subjobok but how to create recovery point in parallel jobs.
regards
Praveen L

Re: Recovery point

willm1 — Mon, 03 Nov 2014 22:21:55 GMT

That's a bit tricky... If one of your parallel streams fails, and you're using something like a tDie - which you should - to kill the job, it will stop all other parallel streams and exit the job. How about re-configuring the job such that each parallel stream is a child job? And you call them from a parent job. In the parent job, you can read the exit code of each child job and do a retry or something else. But that way - hopefully - your child job would be the only one killed by the failure, and not the other streams in other child jobs.
You'd accomplish this set up as follows: ParentJob --> tRunJob --> Child Job - and have as many of these as needed. And - the child job can be the same job for each stream, to which you pass different parameters...

Re: Recovery point

Anonymous — Tue, 04 Nov 2014 03:32:52 GMT

Hi,

I have 5 jobs that are running in parallel. if anyone of the jobs fails i want to start that job only. I can set recovery point on subjobok but how to create recovery point in parallel jobs.
regards

Could you please take a look at document TalendHelpCenter:How to set checkpoints on trigger connections to see if the checkpoints is satisfying your needs?
In addition, could you please show us your job design so that we can know your "parallel" work flow.

Best regards
Sabrina

Re: Recovery point

Anonymous — Tue, 04 Nov 2014 11:59:26 GMT

Hi Xiado

Please see the attached screenshot of my execution plan. In the image you can see In the first level i am running 9 jobs. If any one of the jobs fails out of the 9 i need that only job should then proceed to next level of jobs. rather than running all the first level jobs and the proceed to next level.

thanks
praveen

Re: Recovery point

Anonymous — Tue, 04 Nov 2014 12:03:58 GMT

I didn't understand from your first post that you were using execution plans. Recovery in an execution plan is not currently supported. However, it is planned for 5.6.0 / 6.0.0
You can track this feature here
https://jira.talendforge.org/browse/TDI-29231
Lack of recovery option is why I never use execution plans. I am looking forward to trying this out after it is released.

Re: Recovery point

willm1 — Tue, 04 Nov 2014 16:28:50 GMT

Mike - is this what you're referring to - in 5.5?
https://help.talend.com/search/all?query=Accessing+the+Error+Recovery+Management+page&content-lang=en

Re: Recovery point

Anonymous — Tue, 04 Nov 2014 16:57:55 GMT

Sorry, I misread praveenlingam21's screenshot. Ignore my reference to execution plan, not relevant here.
Regarding error recovery, I have not found a way to make this work with parallel processes when using tParallelize component. A recovery point can only be created onSubJobOk which is not available for each parallel task when using tParallelize component. There are various ways to do this by hand not using a tParallelize, one is outlined by willm in this thread.
Perhaps a new feature request is needed? But I see there already it one, and it was closed with comment " Outside of realistic expectations" see this link. Whose expectations I wonder, not mine certainly.
https://jira.talendforge.org/browse/TDI-24776?jql=text%20~%20%22recovery%22
Perhaps this could be reopened with clear use cases specified.