Issues connecting to Kerberos enabled Cloudera Hadoop Cluster
I am using Talend Studio version 5.6.1. I am trying to read from a flat file and write to HDFS file using tHDFSOutput.
I created the Talend job, did a build and copied the zip file to the Hadoop cluster. when I execute the shell script from the Hadoop cluster I got the below error.
my user id exists on the Hadoop cluster. It is integrated with the LDAP/Active Directory.
I am able to authenticate using my id in Kerberos. I can run the hdfs dfs -ls / commands.
On the cluster side, the value for the property 'dfs.namenode.kerberos.principal' in the file /etc/hadoop/conf/hdfs-site.xml is -
hdfs/_HOST@HADOOP1.WUDIP.COM Below is the error message.
$ ./HDFS_Write_test_run.sh
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
15/05/18 15:53:56 INFO Configuration.deprecation: fs.default.name is deprecated. Instead, use fs.defaultFS
15/05/18 15:53:56 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Exception in component tHDFSOutput_1
java.io.IOException: Login failure for 20000123
@WUDIP.COM from keytab /home/20000123/20000123.keytab
at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:891)
at big_data_demo.hdfs_write_test_0_1.HDFS_Write_test.tFileInputDelimited_1Process(HDFS_Write_test.java:473)
at big_data_demo.hdfs_write_test_0_1.HDFS_Write_test.runJobInTOS(HDFS_Write_test.java:871)
at big_data_demo.hdfs_write_test_0_1.HDFS_Write_test.main(HDFS_Write_test.java:746)
Caused by: javax.security.auth.login.LoginException: java.lang.IllegalArgumentException: Illegal principal name
20000123@WUDIP.COM at org.apache.hadoop.security.User.<init>(User.java:50)
at org.apache.hadoop.security.User.<init>(User.java:43)
at org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule.commit(UserGroupInformation.java:178)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
at javax.security.auth.login.LoginContext.login(LoginContext.java:588)
at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:882)
at big_data_demo.hdfs_write_test_0_1.HDFS_Write_test.tFileInputDelimited_1Process(HDFS_Write_test.java:473)
at big_data_demo.hdfs_write_test_0_1.HDFS_Write_test.runJobInTOS(HDFS_Write_test.java:871)
at big_data_demo.hdfs_write_test_0_1.HDFS_Write_test.main(HDFS_Write_test.java:746)
Caused by: org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No rules applied to 20000123
@WUDIP.COM at org.apache.hadoop.security.authentication.util.KerberosName.getShortName(KerberosName.java:389)
at org.apache.hadoop.security.User.<init>(User.java:48)
... 17 more
at javax.security.auth.login.LoginContext.invoke(LoginContext.java:856)
at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
at javax.security.auth.login.LoginContext.login(LoginContext.java:588)
at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:882)
... 3 more
