Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Save an extra $150 Dec 1–7 with code CYBERWEEK - stackable with early bird savings: Register
cancel
Showing results for 
Search instead for 
Did you mean: 
_AnonymousUser
Specialist III
Specialist III

Problem with HDFS

Hello,
First of all, I want to apologize for my English that is not really good.
Let me explain my problem. It comes from HDFS. Indeed, my objective is to send a file to a HDFS cluster. Then I've ask for all the informations to the administrator, who has given then to me. But, when I try to send/read any data from/to the HDF System, I ALWAYS got this exasperating error as response:
Exception in component tHDFSExist_1
java.io.IOException: Call to /XXX.XXX.XXX.XXX:9000 failed on local exception: java.io.EOFException
at org.apache.hadoop.ipc.Client.wrapException(Client.java:1065)
at org.apache.hadoop.ipc.Client.call(Client.java:1033)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224)
at $Proxy1.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:364)
at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:106)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:208)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:175)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:89)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1310)
at org.apache.hadoop.fs.FileSystem.access$100(FileSystem.java:65)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1328)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:226)
at org.apache.hadoop.fs.FileSystem$1.run(FileSystem.java:103)
disconnected
at org.apache.hadoop.fs.FileSystem$1.run(FileSystem.java:101)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:101)
at talenddemosjava.test_0_1.Test.tHDFSExist_1Process(Test.java:410)
at talenddemosjava.test_0_1.Test.tHDFSConnection_1Process(Test.java:351)
at talenddemosjava.test_0_1.Test.runJobInTOS(Test.java:644)
at talenddemosjava.test_0_1.Test.main(Test.java:509)
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(Unknown Source)
at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:774)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:712)


Does anyone have any idea of the origin of the problem: does it come from TOS, or otherwise is it me who made an error in the configuration?
Labels (3)
11 Replies
_AnonymousUser
Specialist III
Specialist III
Author

Is this issue is fixed. We are facing a similar kind of issue.
I have talend tool installed in windows environment and tried to build hdfsconnection. But not able to successfully connect tried with different  element(thdfsget/thdfsput/thdfsinput/thdfsoutput) but no success.
Tried with HortonWorks distribution and Amazon EMR distribution but failed every time. Please help.
Anonymous
Not applicable

I am facing a similar kind of issue. I tried to put file to HDFS, but show following errors:
Exception in component tHDFSPut_1
java.io.IOException: DataStreamer Exception: 
: org.apache.hadoop.hdfs.DFSClient - DataStreamer Exception
java.nio.channels.UnresolvedAddressException
at sun.nio.ch.Net.checkAddress(Net.java:101)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:622)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530)
at org.apache.hadoop.hdfs.DFSOutputStream.createSocketForPipeline(DFSOutputStream.java:1752)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1530)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1483)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:668)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:796)
Caused by: java.nio.channels.UnresolvedAddressException
at sun.nio.ch.Net.checkAddress(Net.java:101)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:622)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530)
at org.apache.hadoop.hdfs.DFSOutputStream.createSocketForPipeline(DFSOutputStream.java:1752)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1530)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1483)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:668)
Please help!