topic Re: Hadoop Cluster connection to Amazon EMR in Talend Studio

Hadoop Cluster connection to Amazon EMR

Anonymous — Sat, 16 Nov 2024 11:18:13 GMT

Hi,
Has anyone managed to connect successfully to Amazon EMR (apache version 2.4.0) from local open studio for Big Data 5.6.0 install?
I understand you have to create a ssh tunnel config to enable communication to Amazon EMR.
This ssh connection is successfull as the localhost routed can display the name node as below screenshot:

The error occurs when the check services button is clicked.
See below:

The error logs are as below:

Any help here much appreciated.

Thanks,

Re: Hadoop Cluster connection to Amazon EMR

Anonymous — Thu, 05 Mar 2015 13:35:23 GMT

Hello,
It seems like there is a problem to dispay the EMR error message. But I think I have the solution to your problem.
On amazon 2.4.0, the namenode URI has the port 9000, and the resource manager 9022.
Could you try with the following settings:
Namenode: hdfs://localhost:9000
Resource Manage: localhost:9022
if it does not work, could you try with the full URL (don't use the internal one, use the one starting with ec2-XXX):
Namenode: hdfs://ec2-xxx.amazonaws.com:9000
Resource Manage: ec2-xxx.amazonaws.com:9022