Windows Task Scheduler - Talend DI job does not run, but returns success to Windows batch file
Hello everyone
I have a Windows .BAT batch generated by Tqalend DI 6.4.1 file which I run interactively from Command Shell DOS windows and it works.
This is on Windows 10 latest version.
The Talend jobs take several minutes and up to 30 min to run.
When I schedule the same batch file in Windows Task Scheduler the task stars OK but it immediately returns code 0 (i.e. success) and finishes.
The Talend job inside that batch file does not fun at all (i.e. completes immediately, returning success to the .BAT file).
If there is an error returned from the JAR file I can not see it inside scheduler, it gets swallowed and reported as success back to Windows.
Googling around - the only issue I found was due to mapped disk drive letters and network drives using \\server\share\folder.
To the best of my knowledge my package only use C: drive.
It also connects to HTTP, uses tREST, uses AWS cloud.
I put some debugging code in BAT file which creates a local folder after the JAR file is run just to see if the batch file runs until the last line.
It certainly does run, the folder (MKDIR command) gets created immediately after Talend JAR file is run.
So I am certain that the batch file runs but the Talend JAR file exits immediately.
The PATH is correctly set up, because I put compile and run a Hello World Java program immediately before the Talend JAR file and it compiles, creates .class file successfully.
Have anyone encountered this problem? Any ideas what could be the problem?
I searched the Java generated code, there was a reference to the Redshift JDBC driver LOG file which was located on a mapped network drive U:\workspace....
It was something I did not set, Talend DI picked it up from my workspace default settings, so it was a land mine ready to explode, which it did once I run in Windows task scheduler.
Given this happened "behind my back" I did not check this.
I changed the JDBC log file to point to C:\ instead of mapped drive U:\ and it worked.
This is a problem/bug with Java generated by Talend, IMHO.
If Talend Java code not access a mapped drive such as U:\workspace drive running from task scheduler then it must report ERROR code back to the Windows Command shell.
Instead it just exits and it reports exit code 0, success, which task scheduler reports in job history.
Windows task scheduler is also an evil piece of software, because it does not allow the use the mapped disk drive letters but this is the devil we know, it has always been this way.
I hope this helps the next poor unfortunate who is using Windows Task Scheduler with Talend DI Java.
I meet with Redshift JDBC "problem" before, I use for this context parameter - talend_logs+jobName+".csv"
but at least on Linux, I was saw error this in logs
not sure - what information it save, need to test when find time - but You can enable logging for scheduler but at least for debug purpose You can always use
re this: "Yes, my primary Jobs run under Linux (and from TAC and standalone), but I have many tuning on windows, and yes some of them call REST Services, some of them work with AWS"
We are also thinking of running Talend jobs under Linux (considering either TAC and Linux standalone "free" options).
Windows Server costs license money and Linux Server is much cheaper/free and given Talend is all about Java/JVM we are thinking of moving scheduled tasks over to Linux Server OS.
Question: how do you run Talend jobs on Linux standalone, specifically? cron? other ?
Are there any Talend Linux scheduled jobs tips, traps, issues - you can share with us on this forum please?
for Linux cron solution - are there any "traps" or "tips" running Talend jobs that is good to know ?
We develop Talend DI jobs on Windows 10 desktops and then we will migrate the JAR files and the shell batch files to run on scheduled tasks on Linux in production so I am wondering how well this will work ? I will have to deal with file paths/folders/directory names between Windows and Linux, this is a known thing.
