topic Re: HDP connection from windows in Installing and Upgrading

HDP connection from windows

Anonymous — Sat, 16 Nov 2024 08:50:59 GMT

Hi everyone, I installed talend BD on windows 7, and I have a connection to the HDP 2.4, where usually I connect to it through putty ssh. Now, I would like to connect to this server through Talend BD,

I take the configuration information from core-site.xml and hdfs-site.xml.

but usually I receive this problem:

Exception in component tHDFSGet_1 (job1)
org.apache.hadoop.net.ConnectTimeoutException: Call From lg-PC/@ip to @ip:50740 failed on socket timeout exception: org.apache.hadoop.net.ConnectTimeoutException: 20000 millis timeout while waiting for channel to be ready for connect. ch : java.nio.channels.SocketChannel[connection-pending remote=isilon.irit.fr/141.115.102.125:50740]; For more details see:  http://wiki.apache.org/hadoop/SocketTimeout
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
	at java.lang.reflect.Constructor.newInstance(Unknown Source)
	at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:751)
	at org.apache.hadoop.ipc.Client.call(Client.java:1431)
	at org.apache.hadoop.ipc.Client.call(Client.java:1358)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
[statistics] disconnected
	at com.sun.proxy.$Proxy7.getListing(Unknown Source)
	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getListing(ClientNamenodeProtocolTranslatorPB.java:573)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:252)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
	at com.sun.proxy.$Proxy8.getListing(Unknown Source)
	at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:2094)
	at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:2077)
	at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:801)
	at org.apache.hadoop.hdfs.DistributedFileSystem.access$700(DistributedFileSystem.java:106)
	at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:863)
	at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:859)
	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
	at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:859)
	at projet1.job1_0_1.job1.tHDFSGet_1Process(job1.java:665)
	at projet1.job1_0_1.job1.tHDFSConnection_1Process(job1.java:421)
	at projet1.job1_0_1.job1.runJobInTOS(job1.java:1100)
	at projet1.job1_0_1.job1.main(job1.java:949)
Caused by: org.apache.hadoop.net.ConnectTimeoutException: 20000 millis timeout while waiting for channel to be ready for connect. ch : java.nio.channels.SocketChannel[connection-pending remote=isilon.irit.fr/141.115.102.125:50740]
	at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:534)
	at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495)
	at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:612)
	at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:710)
	at org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:373)
	at org.apache.hadoop.ipc.Client.getConnection(Client.java:1493)
	at org.apache.hadoop.ipc.Client.call(Client.java:1397)
	... 23 more
Job job1 terminé à 15:32 17/01/2018.

Re: HDP connection from windows

lojdr — Thu, 18 Jan 2018 09:51:26 GMT

Hello,

Connection timeout is mostly firewall issue. Please check firewall configuration between the lg-PC/172.16.3.125 and isilon.irit.fr:50740 (HDP location and computer with Talend).

Regards

Lojdr

Re: HDP connection from windows

Anonymous — Thu, 18 Jan 2018 11:43:10 GMT

Hello, thanks for your response
I verified the firewall configuration, but it's enabled with talend.
And I added the IP address details in the hosts file. it isn't sufficient ?

Regards
Hana

Re: HDP connection from windows

lojdr — Fri, 19 Jan 2018 14:05:42 GMT

Hello,

Try to connect to specific port at HDP server from W7 via telnet. Are you able to establish the connection?

Regards

Lojdr

Re: HDP connection from windows

Anonymous — Sat, 20 Jan 2018 09:53:36 GMT

Hello,
You have a reason I cannot connect through this port only through 22 which open the ssh. But I have take the port from the core-site.xml of hadoop files .

Regards
Hana

Re: HDP connection from windows

lojdr — Mon, 22 Jan 2018 18:26:39 GMT

I see that your job is trying to connect to

141.115.102.125:50740

Are you able to connect to this IP and port from your computer? If only port 22 is allowed you will have to tunnel it somehow.

Re: HDP connection from windows

Anonymous — Tue, 23 Jan 2018 09:12:31 GMT

My computer is not able to connect to this IP and Port.

Re: HDP connection from windows

lojdr — Tue, 23 Jan 2018 14:23:22 GMT

Hello,
You need to arrange your network to be able to access the port where the service/daemon is running.

Regards
Lojdr

Re: HDP connection from windows

Anonymous — Wed, 27 Jun 2018 05:15:35 GMT

Hello everyone.

I have been trying for about 5 days now to setup a Hadoop connection from Talend installed on my laptop running windows.

The cluster (multi node) is on a Linux box.

I would appreciate any help.

Thanks in advance

Re: HDP connection from windows

Anonymous — Wed, 27 Jun 2018 09:18:48 GMT

Hello,

Are you getting error message when you setup a Hadoop connection from Talend?

Could you please refer to this document about:TalendHelpCenter: Setting up a Hadoop cluster connection?

Best regards

Sabrina

Re: HDP connection from windows

Anonymous — Thu, 28 Jun 2018 02:19:29 GMT

First, when I try to create the Hadoop connection and run "Check Services" it fails with this message.

org.talend.designer.hdfsbrowse.exceptions.HadoopServerException: org.talend.designer.hdfsbrowse.exceptions.HadoopServerException: java.util.concurrent.TimeoutException
at org.talend.designer.hdfsbrowse.hadoop.service.check.AbstractCheckedServiceProvider.checkService(AbstractCheckedServiceProvider.java:57)
at

Second, when I try to run tHDFSExists I get this message

[WARN ]: org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[WARN ]: org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory - The short-circuit local reads feature cannot be used because UNIX Domain sockets are not available on Windows.

Can you tell me please if its possible to connect from Windows to a multi-node cluster on Linux? If yes, can you please point me to any example. My cluster does have an edge node which interacts with the internal nodes.

Thank you.

Re: HDP connection from windows

Anonymous — Tue, 07 Aug 2018 11:09:16 GMT

can anybody tell me if windows is supported in HDP 2.5? Version 2.4.2 is supported and for version 2.5.0 there is no download and information available (yet) at http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0-Win/index.html

Re: HDP connection from windows

Anonymous — Wed, 05 Dec 2018 10:19:14 GMT

Hello @jakemendonza @lykmyk

Sorry for the latest response, I had a lot of problems with HDP, but when I change working with CDP (Cloudera) its works to me without any error.

best regards

hana