topic Find talend job process name at operating system level in Talend Studio

Find talend job process name at operating system level

borrisbeck — Fri, 23 Mar 2018 12:53:51 GMT

I have a job which has to run throughout the day, if it aborts for any reason, it has to be resubmitted automatically, In other ETL tools like Datastage, I would be able to find the ETL job process name at OS level using ps-ef, when i tried that for talend in windows environment using tasklist command, i'm not able to find that Talend job name process which runs at the backed. I know I can create another job which checks if a process is running but I'm trying to create a non Talend based solution and i don't want to export the job and run it outside TAC.. so how can find at OS level if a job is running.

Thanks

Re: Find talend job process name at operating system level

Anonymous — Fri, 23 Mar 2018 13:34:36 GMT

The answer to your question is the following command on the machine that is running your job....

jps -v

On another note, I think you should reconsider your process here. A Talend job is not really suitable to be running all day non-stop. They are meant for batch. You can run a Talend job 1000s of times a day, no problem. If you try to run one continuously for the whole day you will likely end up with memory issues after a while. It is not what Talend DI jobs have been designed for. Your problem sounds to me to be more aligned with a real time (or near to real time) solution. It sounds like a Talend Route (ESB) would be a better fit.

Re: Find talend job process name at operating system level

borrisbeck — Fri, 23 Mar 2018 17:22:48 GMT

Hi
Thanks for responding, to be more specific,I have a main job (Orchestrate job) that has to run from 5.00 AM to 4.30 PM, That orchestrate job triggers a set of 5 jobs sequentially for every 1 hour. The thought behind the question was, I don't want any manual intervention if the main orchestrate job abort, so how do you think I can better design this without the memory issue? please

thanks
MJ

Re: Find talend job process name at operating system level

Anonymous — Fri, 23 Mar 2018 18:54:04 GMT

The first way that springs to mind would not be much of a change to your job structure, but would make it a little more resistant to issues. You could use a Windows scheduler (or Cron on Linux) to run your orchestration job every hour. I assume that the child jobs need to run strictly sequentially (with maybe some interaction with data processed?). As I said, this would not be a massive change for you. The big issue here is solely having a Talend job running as a scheduling device. I assume you have had issues with doing this, hence the original question about monitoring whether it is still running.