Talend don't show Hadoop Resource Manager when I check services

[aazhdmaster1612007186]

Hi community,

I'm trying to configure an Hadoop Connection on Talend Big Data Studio 7.4 using Cloudera Yarn mode Distribution.

On my Hadoop Cluster everything is running fine, and all required ports are opened:

My configuration files are correctly setted because I m able to verify that everithing is running well:

• core-site.xml

<configuration>

<property>

<name>fs.defaultFS</name>

<value>hdfs://ec2-15-237-143-17.eu-west-3.compute.amazonaws.com:9000</value>

</property>

</configuration>

• yarn-site.xml

<configuration>

 <property>

  <name>yarn.nodemanager.aux-services</name>

  <value>mapreduce_shuffle</value>

 </property>

 <property>

  <name>yarn.resourcemanager.hostname</name>

  <value>ec2-15-237-143-17.eu-west-3.compute.amazonaws.com</value>

 </property>

  <property>

  <name>yarn.nodemanager.resource.memory-mb</name>

  <value>768</value>

 </property>

 <property>

  <name>yarn.scheduler.maximum-allocation-mb</name>

  <value>768</value>

 </property>

 <property>

  <name>yarn.scheduler.minimum-allocation-mb</name>

  <value>64</value>

 </property>

 <property>

  <name>yarn.nodemanager.vmem-check-enabled</name>

  <value>false</value>

 </property>​

</configuration>

• mapred-site.xml

<configuration>

 <property>

  <name>mapreduce.jobtracker.address</name>

  <value>ec2-15-237-143-17.eu-west-3.compute.amazonaws.com:54311</value>

 </property>

 <property>

  <name>mapreduce.framework.name</name>

  <value>yarn</value>

 </property>

  <property>

  <name>yarn.app.mapreduce.am.resource.mb</name>

  <value>256</value>

 </property>

 <property>

  <name>mapreduce.map.memory.mb</name>

  <value>128</value>

 </property>

 <property>

  <name>mapreduce.reduce.memory.mb</name>

  <value>128</value>

 </property>​

</configuration>

Then, when I try to check my services with Talend connection wizard, I can only see NodeManager status, but not Resource Manager:

I tried several ports for Resource Manager configuration in Talend (8088, 8032, 8042) but none of them seems to be OK because I can't see the service in Talend service checker.

Please could you advice me or at least give me some hints to go furhter on my analysis.

Thank you very much in advance.

[Anonymous]

Hello , 7.4 is nt a stabe version, don't use it to connect to a production hadoop cluster, i recommand you to use Talend 7.3.1 and check which hadoop distribution are supported;

https://help.talend.com/r/PT_ZIkoUVpsz37AQmeo26w/ObJdkIofVjlRixDgr_gY5Q