Hadoop Cluster Configuration: Cloudera Manager version 5.10.0

Anonymous — Sat, 16 Nov 2024 10:01:34 GMT

Hi Team,
I'm using the BigData Open studio for BigData 6.3.1
My Hadoop Cluster is up and running with Cloudera Manager 5.10.0 managing it. I'm trying to create a metadata connection to the cluster by retrieving configurations from Cloudera Manager.
I'm able to connect to the Cloudera manager from the Talend Metadata Import Wizard, and it fetches the services running on it. In the next step where we have to configure name node and configuration manager URI, I enter the details and check the connection again.
The namenode connection is successful but the configuartion manager connection isn't. This is the error report generated:-

org.talend.designer.hdfsbrowse.exceptions.HadoopServerException: org.talend.designer.hdfsbrowse.exceptions.HadoopServerException: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException
at org.talend.designer.hdfsbrowse.hadoop.service.check.AbstractCheckedServiceProvider.checkService(AbstractCheckedServiceProvider.java:55)
at org.talend.designer.hdfsbrowse.hadoop.service.HadoopServiceBean.check(HadoopServiceBean.java:102)
at org.talend.designer.hdfsbrowse.hadoop.service.check.CheckHadoopServicesDialog$5.run(CheckHadoopServicesDialog.java:373)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.talend.designer.hdfsbrowse.exceptions.HadoopServerException: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException
at org.talend.designer.hdfsbrowse.hadoop.service.check.CheckedWorkUnit.execute(CheckedWorkUnit.java:47)
at org.talend.designer.hdfsbrowse.hadoop.service.check.AbstractCheckedServiceProvider.checkService(AbstractCheckedServiceProvider.java:52)
... 5 more
Caused by: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:206)
at org.talend.designer.hdfsbrowse.hadoop.service.check.CheckedWorkUnit.execute(CheckedWorkUnit.java:44)
... 6 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.talend.core.utils.ReflectionUtils.newInstance(ReflectionUtils.java:278)
at org.talend.designer.hdfsbrowse.hadoop.service.check.provider.CheckedResourceManagerProvider.check(CheckedResourceManagerProvider.java:51)
at org.talend.designer.hdfsbrowse.hadoop.service.check.AbstractCheckedServiceProvider$1.run(AbstractCheckedServiceProvider.java:47)
at org.talend.designer.hdfsbrowse.hadoop.service.check.CheckedWorkUnit$1.call(CheckedWorkUnit.java:65)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Caused by: java.io.IOException: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses.
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:120)
at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:75)
at org.apache.hadoop.mapred.JobClient.init(JobClient.java:472)
at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:450)
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px '.SF NS Text'} span.Apple-tab-span {white-space re}
... 12 more

It's possible that the cluster initialization is failing because I'm telling the Talend wizard that my Cloudera Manager version is 5.8 YARN (It's actually 5.10.0). The jars for 5.8 may not be compatible with 5.10. Is there any way to get the right Cloudera Manager option in the wizard?

Re: Hadoop Cluster Configuration: Cloudera Manager version 5.10.0

Anonymous — Fri, 03 Mar 2017 09:55:30 GMT

Found the solution,
Had to add the IPs and the hostnames to my Mac's /etc/hosts. Talend wizard was then able to resolve the DNS
P.S. You can configure Cloudera Manager 5.8 YARN in the Talend Metadata wizard if you want to use 5.9 and above. It works!

topic Re: Hadoop Cluster Configuration: Cloudera Manager version 5.10.0 in Talend Studio

Hadoop Cluster Configuration: Cloudera Manager version 5.10.0

Re: Hadoop Cluster Configuration: Cloudera Manager version 5.10.0