https://community.qlik.com/t5/Talend-Studio/HDFS-Output-Data-Streamer-Exception/m-p/2263786#M43834 Hello,&nbsp; <BR />I am trying to build a simple job, like in the image below: <BR />The Output file is created on HDFS cluster but no data is written. <BR /> <BR />But I keep having the errors below: <BR />Starting job Test at 09:31 20/12/2016. <BR /> connecting to socket on port 3756 <BR /> connected <BR />: org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable <BR />ID FirstName LastName <BR />1|Warren|Harrison <BR />1|Bill|Eisenhower <BR />1|Andrew|Johnson <BR />1|Bill|Johnson <BR />1|Franklin|Harding <BR />1|Lyndon|Grant <BR />1|Richard|Reagan <BR />1|Benjamin|Wilson <BR />1|George|Quincy <BR />1|Gerald|Harding <BR />1|Franklin|Pierce <BR />1|William|Harrison <BR />1|Grover|Johnson <BR />1|William|Harrison <BR />1|Ulysses|Fillmore <BR />1|Gerald|Fillmore <BR />1|Dwight|Fillmore <BR />1|Jimmy|Johnson <BR />1|Chester|Harrison <BR />1|Richard|Taft <BR />1|Thomas|Carter <BR />1|Lyndon|Hayes <BR />1|Gerald|Harrison <BR />1|Thomas|Roosevelt <BR />1|Andrew|Harrison <BR />1|Thomas|Reagan <BR />1|Theodore|Grant <BR />1|William|Van Buren <BR />1|Dwight|Arthur <BR />1|Dwight|Pierce <BR />1|Chester|Monroe <BR />1|Lyndon|Truman <BR />1|Lyndon|Taft <BR />1|Woodrow|Monroe <BR />1|Woodrow|Johnson <BR />1|Rutherford|Harrison <BR />1|Chester|Grant <BR />1|Rutherford|Washington <BR />1|Woodrow|Pierce <BR />1|Calvin|Reagan <BR />1|James|Buchanan <BR />1|Ulysses|Nixon <BR />1|Gerald|Reagan <BR />1|Lyndon|Wilson <BR />1|George|Johnson <BR />1|Gerald|Adams <BR />1|Rutherford|Reagan <BR />1|Martin|Harrison <BR />1|John|Coolidge <BR />1|Dwight|Washington <BR />1|Theodore|Taft <BR />1|Benjamin|Hoover <BR />1|Millard|Garfield <BR />1|Jimmy|Clinton <BR />1|Chester|Harrison <BR />1|James|Coolidge <BR />1|Warren|Ford <BR />1|George|Buchanan <BR />1|Harry|Adams <BR />1|John|Adams <BR />1|Abraham|Wilson <BR />1|Jimmy|Truman <BR />1|Calvin|Carter <BR />1|Chester|Fillmore <BR />1|Andrew|Fillmore <BR />1|Andrew|Wilson <BR />1|Martin|Tyler <BR />1|Lyndon|Reagan <BR />1|Lyndon|Polk <BR />1|Ulysses|Hayes <BR />1|Franklin|Carter <BR />1|Ronald|Eisenhower <BR />1|Lyndon|Kennedy <BR />1|William|Eisenhower <BR />1|Herbert|McKinley <BR />1|Millard|Jefferson <BR />1|Gerald|Truman <BR />1|Abraham|Hayes <BR />1|Woodrow|Tyler <BR />1|Ulysses|Van Buren <BR />1|James|Harrison <BR />1|Benjamin|Cleveland <BR />1|Lyndon|Roosevelt <BR />1|Calvin|Harding <BR />1|George|Reagan <BR />1|Grover|Lincoln <BR />1|Theodore|Reagan <BR />1|George|Ford <BR />1|Benjamin|Coolidge <BR />1|Martin|Polk <BR />1|John|Jefferson <BR />1|Theodore|Ford <BR />1|Zachary|Wilson <BR />1|Franklin|Monroe <BR />1|Theodore|Truman <BR />1|Herbert|Buchanan <BR />1|Grover|Jackson <BR />1|Abraham|Adams <BR />1|Andrew|Roosevelt <BR />1|Zachary|Hayes <BR />: org.apache.hadoop.hdfs.DFSClient - DataStreamer Exception <BR />java.nio.channels.UnresolvedAddressException <BR />at sun.nio.ch.Net.checkAddress(Unknown Source) <BR />at sun.nio.ch.SocketChannelImpl.connect(Unknown Source) <BR />at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192) <BR />at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530) <BR />at org.apache.hadoop.hdfs.DFSOutputStream.createSocketForPipeline(DFSOutputStream.java:1782) <BR />at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1559) <BR />at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1512) <BR />at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:668) <BR />Exception in component tHDFSOutput_2 <BR />java.io.IOException: DataStreamer Exception:&nbsp; <BR />at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:796) <BR />Caused by: java.nio.channels.UnresolvedAddressException <BR />at sun.nio.ch.Net.checkAddress(Unknown Source) <BR />at sun.nio.ch.SocketChannelImpl.connect(Unknown Source) <BR />at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192) <BR />at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530) <BR />at org.apache.hadoop.hdfs.DFSOutputStream.createSocketForPipeline(DFSOutputStream.java:1782) <BR />at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1559) <BR />at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1512) <BR />at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:668) <BR /> disconnected <BR />: org.apache.hadoop.hdfs.DFSClient - Failed to close inode 5757474 <BR />java.io.IOException: DataStreamer Exception:&nbsp; <BR />at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:796) <BR />Caused by: java.nio.channels.UnresolvedAddressException <BR />at sun.nio.ch.Net.checkAddress(Unknown Source) <BR />at sun.nio.ch.SocketChannelImpl.connect(Unknown Source) <BR />at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192) <BR />at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530) <BR />at org.apache.hadoop.hdfs.DFSOutputStream.createSocketForPipeline(DFSOutputStream.java:1782) <BR />at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1559) <BR />at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1512) <BR />at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:668) <BR />Job Test ended at 09:31 20/12/2016. <BR />HDFS connection looks like this: <BR />Note: If I use the namenode instead of IP , I receive more errors. <BR />Thank you! <BR /> <span class="lia-inline-image-display-wrapper" image-alt="0683p000009MChJ.jpg"><img src="https://community.qlik.com/t5/image/serverpage/image-id/155155i0EA94DFDACA985AC/image-size/large?v=v2&amp;px=999" role="button" title="0683p000009MChJ.jpg" alt="0683p000009MChJ.jpg" /></span> <span class="lia-inline-image-display-wrapper" image-alt="0683p000009MChS.jpg"><img src="https://community.qlik.com/t5/image/serverpage/image-id/131197iF2C7937D17CD83B3/image-size/large?v=v2&amp;px=999" role="button" title="0683p000009MChS.jpg" alt="0683p000009MChS.jpg" /></span> <span class="lia-inline-image-display-wrapper" image-alt="0683p000009MChX.jpg"><img src="https://community.qlik.com/t5/image/serverpage/image-id/147055iEC59E0B3D8BF30A8/image-size/large?v=v2&amp;px=999" role="button" title="0683p000009MChX.jpg" alt="0683p000009MChX.jpg" /></span> Sat, 16 Nov 2024 10:11:33 GMT Anonymous 2024-11-16T10:11:33Z https://community.qlik.com/t5/Talend-Studio/HDFS-Output-Data-Streamer-Exception/m-p/2263786#M43834 Hello,&nbsp; <BR />I am trying to build a simple job, like in the image below: <BR />The Output file is created on HDFS cluster but no data is written. <BR /> <BR />But I keep having the errors below: <BR />Starting job Test at 09:31 20/12/2016. <BR /> connecting to socket on port 3756 <BR /> connected <BR />: org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable <BR />ID FirstName LastName <BR />1|Warren|Harrison <BR />1|Bill|Eisenhower <BR />1|Andrew|Johnson <BR />1|Bill|Johnson <BR />1|Franklin|Harding <BR />1|Lyndon|Grant <BR />1|Richard|Reagan <BR />1|Benjamin|Wilson <BR />1|George|Quincy <BR />1|Gerald|Harding <BR />1|Franklin|Pierce <BR />1|William|Harrison <BR />1|Grover|Johnson <BR />1|William|Harrison <BR />1|Ulysses|Fillmore <BR />1|Gerald|Fillmore <BR />1|Dwight|Fillmore <BR />1|Jimmy|Johnson <BR />1|Chester|Harrison <BR />1|Richard|Taft <BR />1|Thomas|Carter <BR />1|Lyndon|Hayes <BR />1|Gerald|Harrison <BR />1|Thomas|Roosevelt <BR />1|Andrew|Harrison <BR />1|Thomas|Reagan <BR />1|Theodore|Grant <BR />1|William|Van Buren <BR />1|Dwight|Arthur <BR />1|Dwight|Pierce <BR />1|Chester|Monroe <BR />1|Lyndon|Truman <BR />1|Lyndon|Taft <BR />1|Woodrow|Monroe <BR />1|Woodrow|Johnson <BR />1|Rutherford|Harrison <BR />1|Chester|Grant <BR />1|Rutherford|Washington <BR />1|Woodrow|Pierce <BR />1|Calvin|Reagan <BR />1|James|Buchanan <BR />1|Ulysses|Nixon <BR />1|Gerald|Reagan <BR />1|Lyndon|Wilson <BR />1|George|Johnson <BR />1|Gerald|Adams <BR />1|Rutherford|Reagan <BR />1|Martin|Harrison <BR />1|John|Coolidge <BR />1|Dwight|Washington <BR />1|Theodore|Taft <BR />1|Benjamin|Hoover <BR />1|Millard|Garfield <BR />1|Jimmy|Clinton <BR />1|Chester|Harrison <BR />1|James|Coolidge <BR />1|Warren|Ford <BR />1|George|Buchanan <BR />1|Harry|Adams <BR />1|John|Adams <BR />1|Abraham|Wilson <BR />1|Jimmy|Truman <BR />1|Calvin|Carter <BR />1|Chester|Fillmore <BR />1|Andrew|Fillmore <BR />1|Andrew|Wilson <BR />1|Martin|Tyler <BR />1|Lyndon|Reagan <BR />1|Lyndon|Polk <BR />1|Ulysses|Hayes <BR />1|Franklin|Carter <BR />1|Ronald|Eisenhower <BR />1|Lyndon|Kennedy <BR />1|William|Eisenhower <BR />1|Herbert|McKinley <BR />1|Millard|Jefferson <BR />1|Gerald|Truman <BR />1|Abraham|Hayes <BR />1|Woodrow|Tyler <BR />1|Ulysses|Van Buren <BR />1|James|Harrison <BR />1|Benjamin|Cleveland <BR />1|Lyndon|Roosevelt <BR />1|Calvin|Harding <BR />1|George|Reagan <BR />1|Grover|Lincoln <BR />1|Theodore|Reagan <BR />1|George|Ford <BR />1|Benjamin|Coolidge <BR />1|Martin|Polk <BR />1|John|Jefferson <BR />1|Theodore|Ford <BR />1|Zachary|Wilson <BR />1|Franklin|Monroe <BR />1|Theodore|Truman <BR />1|Herbert|Buchanan <BR />1|Grover|Jackson <BR />1|Abraham|Adams <BR />1|Andrew|Roosevelt <BR />1|Zachary|Hayes <BR />: org.apache.hadoop.hdfs.DFSClient - DataStreamer Exception <BR />java.nio.channels.UnresolvedAddressException <BR />at sun.nio.ch.Net.checkAddress(Unknown Source) <BR />at sun.nio.ch.SocketChannelImpl.connect(Unknown Source) <BR />at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192) <BR />at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530) <BR />at org.apache.hadoop.hdfs.DFSOutputStream.createSocketForPipeline(DFSOutputStream.java:1782) <BR />at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1559) <BR />at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1512) <BR />at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:668) <BR />Exception in component tHDFSOutput_2 <BR />java.io.IOException: DataStreamer Exception:&nbsp; <BR />at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:796) <BR />Caused by: java.nio.channels.UnresolvedAddressException <BR />at sun.nio.ch.Net.checkAddress(Unknown Source) <BR />at sun.nio.ch.SocketChannelImpl.connect(Unknown Source) <BR />at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192) <BR />at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530) <BR />at org.apache.hadoop.hdfs.DFSOutputStream.createSocketForPipeline(DFSOutputStream.java:1782) <BR />at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1559) <BR />at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1512) <BR />at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:668) <BR /> disconnected <BR />: org.apache.hadoop.hdfs.DFSClient - Failed to close inode 5757474 <BR />java.io.IOException: DataStreamer Exception:&nbsp; <BR />at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:796) <BR />Caused by: java.nio.channels.UnresolvedAddressException <BR />at sun.nio.ch.Net.checkAddress(Unknown Source) <BR />at sun.nio.ch.SocketChannelImpl.connect(Unknown Source) <BR />at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192) <BR />at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530) <BR />at org.apache.hadoop.hdfs.DFSOutputStream.createSocketForPipeline(DFSOutputStream.java:1782) <BR />at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1559) <BR />at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1512) <BR />at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:668) <BR />Job Test ended at 09:31 20/12/2016. <BR />HDFS connection looks like this: <BR />Note: If I use the namenode instead of IP , I receive more errors. <BR />Thank you! <BR /> <span class="lia-inline-image-display-wrapper" image-alt="0683p000009MChJ.jpg"><img src="https://community.qlik.com/t5/image/serverpage/image-id/155155i0EA94DFDACA985AC/image-size/large?v=v2&amp;px=999" role="button" title="0683p000009MChJ.jpg" alt="0683p000009MChJ.jpg" /></span> <span class="lia-inline-image-display-wrapper" image-alt="0683p000009MChS.jpg"><img src="https://community.qlik.com/t5/image/serverpage/image-id/131197iF2C7937D17CD83B3/image-size/large?v=v2&amp;px=999" role="button" title="0683p000009MChS.jpg" alt="0683p000009MChS.jpg" /></span> <span class="lia-inline-image-display-wrapper" image-alt="0683p000009MChX.jpg"><img src="https://community.qlik.com/t5/image/serverpage/image-id/147055iEC59E0B3D8BF30A8/image-size/large?v=v2&amp;px=999" role="button" title="0683p000009MChX.jpg" alt="0683p000009MChX.jpg" /></span> Sat, 16 Nov 2024 10:11:33 GMT https://community.qlik.com/t5/Talend-Studio/HDFS-Output-Data-Streamer-Exception/m-p/2263786#M43834 Anonymous 2024-11-16T10:11:33Z https://community.qlik.com/t5/Talend-Studio/HDFS-Output-Data-Streamer-Exception/m-p/2263787#M43835 Hi,<BR /><FONT size="1"><FONT face="Verdana, Helvetica, Arial, sans-serif">Can you confirm that the machine you are running the job on can access your HDFS connection successfully? A</FONT></FONT><FONT size="1"><FONT face="Verdana, Helvetica, Arial, sans-serif">ll ports have been opened in the firewall, and can you make a telnet to that port successfully?</FONT></FONT><BR /><FONT size="1"><FONT face="Verdana, Helvetica, Arial, sans-serif">Have you tried to use hostname of your namenode in your HDFSOutput component?&nbsp;</FONT></FONT><BR /><FONT size="1"><FONT face="Verdana, Helvetica, Arial, sans-serif">Best regards</FONT></FONT><BR /><FONT size="1"><FONT face="Verdana, Helvetica, Arial, sans-serif">Sabrina</FONT></FONT> Wed, 21 Dec 2016 07:02:11 GMT https://community.qlik.com/t5/Talend-Studio/HDFS-Output-Data-Streamer-Exception/m-p/2263787#M43835 Anonymous 2016-12-21T07:02:11Z https://community.qlik.com/t5/Talend-Studio/HDFS-Output-Data-Streamer-Exception/m-p/2263788#M43836 Hi Sabrina,&nbsp; <BR />I have telnet succesfully port 8022,8020. <BR />When I use hostname of my namenode, in both of my HDFS components, I get the errors above: <BR />&nbsp;Starting job Test at 14:36 21/12/2016. <BR /> connecting to socket on port 3839 <BR /> connected <BR />: org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable <BR />Exception in component tHDFSOutput_2 <BR />java.lang.IllegalArgumentException: java.net.UnknownHostException: namenode1.hcluster.su.ro.conti.de <BR /> at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:374) <BR /> at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:310) <BR /> at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176) <BR /> at org.apache.hadoop.hdfs.DFSClient.&lt;init&gt;(DFSClient.java:708) <BR /> at org.apache.hadoop.hdfs.DFSClient.&lt;init&gt;(DFSClient.java:651) <BR /> at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149) <BR /> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2696) <BR /> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94) <BR /> at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2733) <BR /> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2715) <BR /> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:382) <BR /> at org.apache.hadoop.fs.FileSystem$1.run(FileSystem.java:171) <BR /> at org.apache.hadoop.fs.FileSystem$1.run(FileSystem.java:168) <BR /> at java.security.AccessController.doPrivileged(Native Method) <BR /> at javax.security.auth.Subject.doAs(Unknown Source) <BR /> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) <BR /> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:168) <BR /> at helloworld.test_0_1.Test.tRowGenerator_3Process(Test.java:971) <BR /> at helloworld.test_0_1.Test.tHDFSConnection_1Process(Test.java:400) <BR /> at helloworld.test_0_1.Test.runJobInTOS(Test.java:1538) <BR /> at helloworld.test_0_1.Test.main(Test.java:1395) <BR />Caused by: java.net.UnknownHostException: namenode1.hcluster.su.ro.conti.de <BR /> ... 21 more <BR /> disconnected <BR />Job Test ended at 14:36 21/12/2016. Wed, 21 Dec 2016 12:39:45 GMT https://community.qlik.com/t5/Talend-Studio/HDFS-Output-Data-Streamer-Exception/m-p/2263788#M43836 Anonymous 2016-12-21T12:39:45Z https://community.qlik.com/t5/Talend-Studio/HDFS-Output-Data-Streamer-Exception/m-p/2263789#M43837 Hi, <BR />Have you already checked document about: <A href="https://help.talend.com/search/all?query=Setting+up+Hadoop+connection+manually&amp;content-lang=en" target="_blank" rel="nofollow noopener noreferrer">TalendHelpCenter<span class="lia-inline-image-display-wrapper" image-alt="0683p000009M9p6.png"><img src="https://community.qlik.com/t5/image/serverpage/image-id/134116iFBD5D7F21624A744/image-size/large?v=v2&amp;px=999" role="button" title="0683p000009M9p6.png" alt="0683p000009M9p6.png" /></span>etting up Hadoop connection manually</A>? <BR />Best regards <BR />Sabrina Thu, 22 Dec 2016 07:52:17 GMT https://community.qlik.com/t5/Talend-Studio/HDFS-Output-Data-Streamer-Exception/m-p/2263789#M43837 Anonymous 2016-12-22T07:52:17Z