topic Re: Error log4j when execute a job with Crontab in Talend Studio

Error log4j when execute a job with Crontab

Anonymous — Wed, 16 May 2018 14:20:24 GMT

Hi guys,

I need yours help. I have a job in Talend, this job works. Then i need to excute with Crontab in a Linux Server.

When i executed manually works but with crontab i have this error:

log4j:WARN No appenders could be found for logger (org.apache.hive.jdbc.Utils).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

I don't know what happened. I read that i need to active the log4j but i have a free Talend version. Please help me! I attached one photo of my job. Thank you!

Re: Error log4j when execute a job with Crontab

Jesperrekuh — Wed, 16 May 2018 21:11:18 GMT

Can't recall if I had exactly the same... But the first two errors you got in the crontab is something with your eclipse properties at runtime (which works wel) vs builded job... added a screenshot which worked for me.

Imported log4j.jar ... download if you cant find it at Log4j Jar

log4j.jpg

Re: Error log4j when execute a job with Crontab

Anonymous — Wed, 16 May 2018 21:38:38 GMT

Thank you!

But i have a error when i schedule the job in crontab:

LOG_CAPTURE_run.sh: line 4: java: command not found

I don't know to resolved this error. When i revised the line 4:

java -Xms256M -Xmx1024M -cp .:$ROOT_PATH:$ROOT_PATH/../lib/routines.jar:$ROOT_PATH/../lib/commons-collections-3.2.2.jar:$ROOT_PATH/../lib/commons-lang-2.6.jar:$ROOT_PATH/../lib/log4j-1.2.16.jar:$ROOT_PATH/../lib/log4j-1.2.17.jar:$ROOT_PATH/../lib/slf4j-api-1.7.5.jar:$ROOT_PATH/../lib/slf4j-log4j12-1.7.5.jar:$ROOT_PATH/../lib/dom4j-1.6.1.jar:$ROOT_PATH/../lib/jackson-xc-1.8.8.jar:$ROOT_PATH/../lib/hadoop-conf-NEW.jar:$ROOT_PATH/../lib/hadoop-yarn-api-2.6.0-cdh5.8.1.jar:$ROOT_PATH/../lib/httpclient-4.2.5.jar:$ROOT_PATH/../lib/jackson-mapper-asl-1.8.8.jar:$ROOT_PATH/../lib/hadoop-auth-2.6.0-cdh5.8.1.jar:$ROOT_PATH/../lib/jersey-client-1.9.jar:$ROOT_PATH/../lib/jackson-core-asl-1.8.8.jar:$ROOT_PATH/../lib/antlr-runtime-3.4.jar:$ROOT_PATH/../lib/avro-1.7.6-cdh5.8.1.jar:$ROOT_PATH/../lib/commons-cli-1.2.jar:$ROOT_PATH/../lib/datanucleus-core-3.2.10.jar:$ROOT_PATH/../lib/hadoop-yarn-common-2.6.0-cdh5.8.1.jar:$ROOT_PATH/../lib/hive-jdbc-1.1.0-cdh5.8.1.jar:$ROOT_PATH/../lib/hive-metastore-1.1.0-cdh5.8.1.jar:$ROOT_PATH/../lib/hive-exec-1.1.0-cdh5.8.1.jar:$ROOT_PATH/../lib/commons-configuration-1.6.jar:$ROOT_PATH/../lib/jdo-api-3.0.1.jar:$ROOT_PATH/../lib/curator-framework-2.6.0.jar:$ROOT_PATH/../lib/commons-codec-1.9.jar:$ROOT_PATH/../lib/httpcore-4.3.3.jar:$ROOT_PATH/../lib/htrace-core-3.2.0-incubating.jar:$ROOT_PATH/../lib/commons-io-2.4.jar:$ROOT_PATH/../lib/jackson-jaxrs-1.8.8.jar:$ROOT_PATH/../lib/derby-10.11.1.1.jar:$ROOT_PATH/../lib/hadoop-hdfs-2.6.0-cdh5.8.1.jar:$ROOT_PATH/../lib/libthrift-0.9.2.jar:$ROOT_PATH/../lib/servlet-api-2.5.jar:$ROOT_PATH/../lib/datanucleus-rdbms-3.2.9.jar:$ROOT_PATH/../lib/httpcore-4.2.5.jar:$ROOT_PATH/../lib/hadoop-mapreduce-client-common-2.6.0-cdh5.8.1.jar:$ROOT_PATH/../lib/hadoop-yarn-client-2.6.0-cdh5.8.1.jar:$ROOT_PATH/../lib/mssql-jdbc.jar:$ROOT_PATH/../lib/hadoop-mapreduce-client-core-2.6.0-cdh5.8.1.jar:$ROOT_PATH/../lib/commons-logging-1.2.jar:$ROOT_PATH/../lib/guava-12.0.1.jar:$ROOT_PATH/../lib/hadoop-yarn-server-web-proxy-2.6.0-cdh5.8.1.jar:$ROOT_PATH/../lib/hive-service-1.1.0-cdh5.8.1.jar:$ROOT_PATH/../lib/libfb303-0.9.2.jar:$ROOT_PATH/../lib/httpclient-4.3.3.jar:$ROOT_PATH/../lib/hadoop-common-2.6.0-cdh5.8.1.jar:$ROOT_PATH/../lib/commons-httpclient-3.0.1.jar:$ROOT_PATH/../lib/curator-client-2.6.0.jar:$ROOT_PATH/../lib/jersey-core-1.9.jar:$ROOT_PATH/../lib/htrace-core4-4.0.1-incubating.jar:$ROOT_PATH/../lib/hadoop-mapreduce-client-jobclient-2.6.0-cdh5.8.1.jar:$ROOT_PATH/../lib/talendcsv.jar:$ROOT_PATH/../lib/datanucleus-api-jdo-3.2.6.jar:$ROOT_PATH/../lib/protobuf-java-2.5.0.jar:$ROOT_PATH/../lib/talend_file_enhanced_20070724.jar:$ROOT_PATH/../lib/talend_DB_mssqlUtil.jar:$ROOT_PATH/../lib/zookeeper-3.4.5-cdh5.8.1.jar:$ROOT_PATH/extract_datalog2_log_capture_0_1.jar: local_project.extract_datalog2_log_capture_0_1.Extract_datalog2_LOG_CAPTURE --context=Default "$@"

Somebody can help me?

Re: Error log4j when execute a job with Crontab

Jesperrekuh — Wed, 16 May 2018 23:21:10 GMT

you have to run the <jobname>.sh /full/path/to<jobname>.jar
I had to same due to failed attempt...

Re: Error log4j when execute a job with Crontab

Anonymous — Thu, 17 May 2018 14:51:25 GMT

Hi. Thanks for your answer. But can you explane this?

I don't know where i have to run the job. This it's my route:

sh /home/Victor_Acosta/Datalog2_test_etl/Extract_datalog2/Extract_datalog2_run.sh >> /home/Victor_Acosta/logs/datalog2_daily/Datalog2_test/crontab_datalog2_test.log

Re: Error log4j when execute a job with Crontab

Jesperrekuh — Thu, 17 May 2018 22:49:54 GMT

LOG_CAPTURE_run.sh: line 4: java: command not found

I forgot to read properly... Im sorry, but check this first... I assumed you checked, but It doesnt find java

Run this you should get a reply

[talend@dstage talend]$ java -version
openjdk version "1.8.0_171"
OpenJDK Runtime Environment (build 1.8.0_171-b10)
OpenJDK 64-Bit Server VM (build 25.171-b10, mixed mode)

If because your Victor_Acosta user environment isn't setting environment variables.

so JAVA HOME has not been set or pointing to wrong folder.

Run this before you run your script, if you dont get a line...

[talend@dstage talend]$ env |grep JAVA_HOME
JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.171-7.b10.el7.x86_64/jre

If not add this to $HOME/.bashrc or make sure within the correct $HOME/.bashrc_profile
check for your .bash profile files:

[talend@dstage talend]$ ls -als $HOME
20 -rw-------. 1 talend talend 16558 May 11 21:51 .bash_history
 4 -rw-r--r--. 1 talend talend 18 Sep 26 2017 .bash_logout
 4 -rw-r--r--. 1 talend talend 193 Sep 26 2017 .bash_profile
 4 -rw-r--r--. 1 talend talend 513 May 1 01:23 .bashrc
 4 drwxrwxr-x. 3 talend talend 4096 Apr 20 16:23 .cache

[talend@dstage talend]$ cat $HOME/.bashrc |grep export
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.171-7.b10.el7.x86_64/jre
export PATH=$JAVA_HOME/bin:$PATH

[talend@dstage talend]$ vi $HOME/.bashrc
add your export vars save the file and exit
[talend@dstage talend]$ . $HOME/.bashrc
this will set your export vars

Last thing, if set correctly :

[talend@dstage talend]$ java -version
openjdk version "1.8.0_171"
OpenJDK Runtime Environment (build 1.8.0_171-b10)
OpenJDK 64-Bit Server VM (build 25.171-b10, mixed mode)

Next : Check your if is this line in your Extract_datalog2_run.sh script?

[talend@dstage talend]$ cat Extract_datalog2_run.sh |grep cd
cd `dirname $0`

change it to :

[talend@dstage talend]$ vi Extract_datalog2_run.sh
cd `dirname -- "$0"`

Good luck

Re: Error log4j when execute a job with Crontab

Anonymous — Thu, 17 May 2018 23:08:25 GMT

Wow! Thanks for your answer. I followed your step and I could execute my job using crontab.

Re: Error log4j when execute a job with Crontab

Jesperrekuh — Thu, 17 May 2018 23:42:23 GMT

Honestly, you marked the wrong answer as a solution... did you add the log4j.jar as I suggested?
Because that solved the problem... the other one is regardless of log4j ... just environment setup.

Happy to help!

Re: Error log4j when execute a job with Crontab

Anonymous — Fri, 18 May 2018 14:57:54 GMT

It's true . I can't resolved the problem with log4j but it's just a Warn