topic [resolved] Oozie cannot upload jar files to HDFS. in Talend Studio

[resolved] Oozie cannot upload jar files to HDFS.

Anonymous — Sat, 16 Nov 2024 10:12:16 GMT

Hi,
I have a job which puts (using thdfsoutput) some data to a remote server hdfs. The job works correctly using the standard "Run" tab. No connection problems. When I run the job using oozie I receive the error:

The local file can not upload to Hadoop HDFS!

Unable to upload file C:\...\TOS_BD-20161026_1219-V6.3.0\temp\Twitter\lib\hadoop-hdfs-2.7.1.2.4.0.0-169.jar to path/to/hdfs
What I see in the hdfs directory where the job is being deployed is a bunch of 0-byte jar files...
Any idea of what can be the reason? locally I use jre 1.8, the server OS is running jre 1.7, can this be the problem?
(more info: I can successfully put this jar file to hdfs via Hue.)
Thank you!

Re: [resolved] Oozie cannot upload jar files to HDFS.

Anonymous — Fri, 16 Dec 2016 06:53:10 GMT

Hi,
Big Data Distributions are compatible with Java 1.8 used. Here is the document about:TalendHelpCenter:Java
Could you please try to use JDK 1.8 to see if it works?
Best regards
Sabrina

Re: [resolved] Oozie cannot upload jar files to HDFS.

Anonymous — Fri, 16 Dec 2016 17:06:25 GMT

errorlog.txt
Hi Sabrina,
thank you very much for the answer. After some pain I resolved the java version and now jdk1.8 are installed in all the nodes.
Unfortunately I continue with the same error when I try to run the job from the oozie view. Any idea?
Something I should have done earlier is to click the "check services" button from the edit cluster metadata prompt. Namenode is OK but Resource Manager fails. Hostnames and ports are coherent with the cluster configuration. Does anyone have an idea of what can be causing this error? I attach the error in txt file. It seems to be this error: , but I kerberos security is not enabled in the cluster and I have not ticked this checkbox mentioned in the link.
Please help! Thanks!
errorlog.txt

Re: [resolved] Oozie cannot upload jar files to HDFS.

Anonymous — Mon, 19 Dec 2016 08:44:44 GMT

Hi,
Did you use IP address? Could you please show us your job setting screenshots?(blur your sensitive data)
Best regards
Sabrina

Re: [resolved] Oozie cannot upload jar files to HDFS.

Anonymous — Mon, 19 Dec 2016 09:13:25 GMT

Hi Sabrina,
I resolved the problem. I had a wrong IP address in my hosts file. Thank you for your quick responses.
How can I mark this post as resolved?
Thanks!!!
Pau

Re: [resolved] Oozie cannot upload jar files to HDFS.

Anonymous — Tue, 20 Dec 2016 03:05:08 GMT

Hi,
Great this issue has been fixed. You can click the "Set this topic as resolved" link which is right underneath your initial post to mark this topic as resolved.
Best regards
Sabrina