HDP connection from windows

[Anonymous]

Hi everyone, I installed talend BD on windows 7, and I have a connection to the HDP 2.4, where usually I connect to it through putty ssh. Now, I would like to connect to this server through Talend BD, 

 I take the configuration information from core-site.xml and hdfs-site.xml.

but usually I receive this problem:

Exception in component tHDFSGet_1 (job1)
org.apache.hadoop.net.ConnectTimeoutException: Call From lg-PC/@ip to @ip:50740 failed on socket timeout exception: org.apache.hadoop.net.ConnectTimeoutException: 20000 millis timeout while waiting for channel to be ready for connect. ch : java.nio.channels.SocketChannel[connection-pending remote=isilon.irit.fr/141.115.102.125:50740]; For more details see:  http://wiki.apache.org/hadoop/SocketTimeout
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
	at java.lang.reflect.Constructor.newInstance(Unknown Source)
	at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:751)
	at org.apache.hadoop.ipc.Client.call(Client.java:1431)
	at org.apache.hadoop.ipc.Client.call(Client.java:1358)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
[statistics] disconnected
	at com.sun.proxy.$Proxy7.getListing(Unknown Source)
	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getListing(ClientNamenodeProtocolTranslatorPB.java:573)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:252)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
	at com.sun.proxy.$Proxy8.getListing(Unknown Source)
	at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:2094)
	at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:2077)
	at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:801)
	at org.apache.hadoop.hdfs.DistributedFileSystem.access$700(DistributedFileSystem.java:106)
	at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:863)
	at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:859)
	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
	at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:859)
	at projet1.job1_0_1.job1.tHDFSGet_1Process(job1.java:665)
	at projet1.job1_0_1.job1.tHDFSConnection_1Process(job1.java:421)
	at projet1.job1_0_1.job1.runJobInTOS(job1.java:1100)
	at projet1.job1_0_1.job1.main(job1.java:949)
Caused by: org.apache.hadoop.net.ConnectTimeoutException: 20000 millis timeout while waiting for channel to be ready for connect. ch : java.nio.channels.SocketChannel[connection-pending remote=isilon.irit.fr/141.115.102.125:50740]
	at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:534)
	at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495)
	at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:612)
	at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:710)
	at org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:373)
	at org.apache.hadoop.ipc.Client.getConnection(Client.java:1493)
	at org.apache.hadoop.ipc.Client.call(Client.java:1397)
	... 23 more
Job job1 terminé à 15:32 17/01/2018. 

[lojdr]

Hello,

Connection timeout is mostly firewall issue. Please check firewall configuration between the lg-PC/172.16.3.125 and isilon.irit.fr:50740 (HDP location and computer with Talend).

Regards

Lojdr

[Anonymous]

Hello, thanks for your response
I verified the firewall configuration, but it's enabled with talend.
And I added the IP address details in the hosts file. it isn't sufficient ?

Regards
Hana

[lojdr]

Hello,

Try to connect to specific port at HDP server from W7 via telnet. Are you able to establish the connection?

Regards

Lojdr

[Anonymous]

Hello,
You have a reason I cannot connect through this port only through 22 which open the ssh. But I have take the port from the core-site.xml of hadoop files .

Regards
Hana

[lojdr]

I see that your job is trying to connect to 

141.115.102.125:50740

Are you able to connect to this IP and port from your computer? If only port 22 is allowed you will have to tunnel it somehow.

[Anonymous]

My computer is not able to connect to this IP and Port.

[lojdr]

Hello,
You need to arrange your network to be able to access the port where the service/daemon is running.

Regards
Lojdr

[Anonymous]

Hello everyone.

I have been trying for about 5 days now to setup a Hadoop connection from Talend installed on my laptop running windows.

The cluster (multi node) is on a Linux box. 

I would appreciate any help.

Thanks in advance

[Anonymous]

Hello,

Are you getting error message when you setup a Hadoop connection from Talend?

Could you please refer to this document about:TalendHelpCenter: Setting up a Hadoop cluster connection?

Best regards

Sabrina