topic Re: Schedule talend job using cron tab in Talend Studio

Schedule talend job using cron tab

karthik_T — Wed, 25 Jul 2018 13:41:52 GMT

Hi,

i want to schedule the talend job (.sh) file using cron tab by creating dependencies between jobs like after successful execution of one job only other job should execute. Can any one tell me how to do it.

Re: Schedule talend job using cron tab

Jesperrekuh — Wed, 25 Jul 2018 14:12:09 GMT

Use this with nohup to start in background and return to your prompt:

nohup sh <yourjob>.sh --context_param acontextvariable1='2018-07-21' --context_param acontextvariable2='2018-07-21' --context_param acontextvariable3='clientstable' > /data/log/<yourlogfilename>.log 2>&1 </dev/null &

Re: Schedule talend job using cron tab

karthik_T — Wed, 25 Jul 2018 15:13:17 GMT

Hi Dijke,

Thanks for your reply. i want to create dependency between two jobs after success of one job only other job should execute otherwise should not execute.

can you explain the command you posted.

Re: Schedule talend job using cron tab

Jesperrekuh — Wed, 25 Jul 2018 15:53:00 GMT

I've already explained the command...
nohup -> background task
sh -> execute <yourjob>.sh
--context_param -> pass value to a context variable like cronStartDate='2018-07-21'
context vars you need to set in your job in order to pass your crontab parameters, in example your startdate and an enddate.
> /data/log/<yourlogfilename>.log 2>&1 & -> root stdout and error to a log file and return to unix prompt

For understanding crontab (Cron Table), google it please to get a better understanding also to implement dependencies.

However, a sidenote: I prefer to create my dependencies (provenance) in my job flow not with crontab.
And pass log events (tlogcatcher) to my elastic search environment and visualize it with Kibana.
The generated logfile (from above) is also picked up by logstash and inserted into eleastic search...

Dont integrate to deeply with crontab... keep it at just triggering a job. But it highly depends on your daily operations and maintenance strategy/policy by your company.

Look at ELK stack : ElasticSearch Logstash Kibana., not a scheduling tool and has nothing to do with it.