How to Stop a job when Execution Time is too long

[Anonymous]

I have a job who make something, and i want to add the rule :

If the job duration > 10seconds, stop it.

To get the start duration, i think tChronometerStart can be a good choice because he have : (Long)globalMap.get("tChronometerStart_1_STARTIME")

The problem is the tChronometerStop, because this component help when we want to get the duration of the full job AFTER execution.

I don't know how to write correctly the "stop" duration because after running 10seconds, i want to stop the job.

To stop the job, i think about a tDie with a "if" condition like this : ((Long)globalMap.get("tChronometerStop_2_STARTIME") - (Long)globalMap.get("CURRENT_TIME???"))/1000 > 10 secondes

thanks

[Anonymous]

You are going to have to thin about this in a different way. Can you show us a screenshot of your job? What you will likely need to do is add checks to the job processing so that the elapsed time can be checked in the flow of the job. This is because of the flow of processing within Talend Jobs. 

Show us your job and we can suggest where and how to implement this

[Anonymous]

my job :

if condition : ((Long)globalMap.get("tChronometerStart_1_STARTIME") - (Long)globalMap.get("tChronometerStop_1_DURATION"))/1000 > 10

how can i add checks to the job processing?

[Anonymous · Accepted solution]

OK, what I would do is the following....

1) Use a tPreJob to link to a tJava component. Calculate the current time and set it to the globalMap

2) In your tMap create a tMap boolean variable (between the input and output tables) and use the logic similar to what you have shown in your description. Use your start time globalMap value and compare to the current time (use the TalendDate routine). If the time difference is within 10 secs, set to True, else set to False.

3) Create another tMap output for when the time has expired. Use the tMap output filtering to decide which output to use based on the tMap variable result. For the output when you have run out of time, add any columns you may wish to report have not been done in time and connect that output to whatever component will fulfill your logging. Then connect that component to a tDie using a RunIf with some logic to identify that you do indeed want to kill the process (this is required because outputs from tMap components are always triggered even if there is no data).

By the way, I am assuming you are using the Open Source Edition and cannot use parallel processing. If you are using the Enterprise Edition you can use a tParallelize component and carry out some check based on a tLoop. The reason I have shown the way above is that you need to build the check into the main body of your processing if you want it actioned when essentially running on a single thread

[Anonymous]

it works, thanks

[Anonymous]

No problem. Glad to help

[Dewey]

Why not just use a tSleep that triggers the tDie when it times out?

[Anonymous]

for tSleep, i think it's a solution, but only if you have "tParrallelize" component : one "Thread" who work, one "Thread" who tSleep and stop.
but this component is not available in open studio talend version.

[Anonymous]

I had a similar scenario where i need to end a job automatically when its execution goes beyond 2 minutes ,

Add a tjavaflex in the main flow before tmap and put the condition (as per your time limit) as in the screenshot  ,

Tjavaflex details :

start code - Initialize a variable that stores the starting time .

Main code - An IF Condition checking whether the difference between the stored variable time and the current time is greater than the time limit (5 sec in my case for testing) , at one point of the main flow this becomes true and the break statement inside it is called which stops the code flow. Also have a context flag say , context.timeout for which flag value 1 is assigned inside the IF condition 

Run-IF - use the condition context.timeout==1 , to trigger tDIE so that u can display a timeout error message

Test Scenario :

In the below attached job screenshot Row generator generates 1 crore records. Since i have given a limit of 5 seconds in tjavaflex , execution stops in the middle (at 45 lakh records) and tdie is triggered.

Row generator generates 1 crore records. Since i have given a limit of 5 seconds in tjavaflex , execution stops in the middle and tdie is triggered.
TjavaFlex Code

[Dewey]

I do that in a scheduler I wrote for open DI.

Allow parallel processing in your job. Then include a tSleep component that calls tDie. You will also need a tDie at the end of the actual process since if it ends before the timeout you don't want to wait on the tSleep.

TimeoutAndDie.jpg