Talend Standard Job: Error while loading data into HDFS

RashmiBector — Sat, 16 Nov 2024 01:30:07 GMT

Job Name: J001_Test

JobDescription: tHiveInput --> tHDFSOutput (With tHDFSConfiguration)

Type of Job: Standard job

Error in Talend (Right click on the job --> Run Job):

[statistics] connecting to socket on port 3783

[statistics] connected

[WARN ] 09:42:21 org.apache.hadoop.util.Shell- Did not find winutils.exe: java.io.FileNotFoundException: java.io.FileNotFoundException: HADOOP_HOME and hadoop.home.dir are unset. -see https://wiki.apache.org/hadoop/WindowsProblems

[WARN ] 09:42:21 org.apache.hadoop.util.NativeCodeLoader- Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

Exception in component tHDFSConnection_1 (J001_Test)

org.apache.hadoop.security.KerberosAuthException: failure to login: for principal: svc_bd_tr_tdm_r@DATA.LAKE from keytab /home/at/talenduser/svc_bd_tr_tdm_r.keytab javax.security.auth.login.LoginException: Unable to obtain password from user

at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1993)

at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1361)

at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation

UserGroupInformation.java:1141)

at datafactorytdm.j001_test_0_1.J001_Test.tHDFSConnection_1Process(J001_Test.java:1312)

at datafactorytdm.j001_test_0_1.J001_Test.runJobInTOS(J001_Test.java:3648)

at datafactorytdm.j001_test_0_1.J001_Test.main(J001_Test.java:2513)

Caused by: javax.security.auth.login.LoginException: Unable to obtain password from user

at com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:901)

at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:764)

at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:618)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)

at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)

at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)

at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)

at javax.security.auth.login.LoginContext.login(LoginContext.java:587)

at org.apache.hadoop.security.UserGroupInformation$HadoopLoginContext.login(UserGroupInformation.java:2072)

at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1983)

... 5 more

[FATAL] 09:42:22 datafactorytdm.j001_test_0_1.J001_Test- tHDFSConnection_1 failure to login: for principal: svc_bd_tr_tdm_r@DATA.LAKE from keytab /home/at/talenduser/svc_bd_tr_tdm_r.keytab javax.security.auth.login.LoginException: Unable to obtain password from user

from user

at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1993) ~[hive-jdbc-3.1.3000.7.2.2.0-9-standalone-6.0.0-SNAPSHOT.jar:3.1.3000.7.2.2.0-9]

at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1361) ~[hive-jdbc-3.1.3000.7.2.2.0-9-standalone-6.0.0-SNAPSHOT.jar:3.1.3000.7.2.2.0-9]

at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:1141) ~[hive-jdbc-3.1.3000.7.2.2.0-9-standalone-6.0.0-SNAPSHOT.jar:3.1.3000.7.2.2.0-9]

at datafactorytdm.j001_test_0_1.J001_Test.tHDFSConnection_1Process(J001_Test.java:1312) [classes/:?]

at datafactorytdm.j001_test_0_1.J001_Test.runJobInTOS(J001_Test.java:3648) [classes/:?]

at datafactorytdm.j001_test_0_1.J001_Test.main(J001_Test.java:2513) [classes/:?]

Caused by: javax.security.auth.login.LoginException: Unable to obtain password from user

at com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:901) ~[?:1.8.0_252]

at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:764) ~[?:1.8.0_252]

at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:618) ~[?:1.8.0_252]

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_252]

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_252]

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_252]

at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_252]

at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755) ~[?:1.8.0_252]

at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195) ~[?:1.8.0_252]

at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682) ~[?:1.8.0_252]

at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680) ~[?:1.8.0_252]

at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_252]

at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680) ~[?:1.8.0_252]

at javax.security.auth.login.LoginContext.login(LoginContext.java:587) ~[?:1.8.0_252]

at org.apache.hadoop.security.UserGroupInformation$HadoopLoginContext.login(UserGroupInformation.java:2072) ~[hive-jdbc-3.1.3000.7.2.2.0-9-standalone-6.0.0-SNAPSHOT.jar:3.1.3000.7.2.2.0-9]

at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1983) ~[hive-jdbc-3.1.3000.7.2.2.0-9-standalone-6.0.0-SNAPSHOT.jar:3.1.3000.7.2.2.0-9]

... 5 more

[statistics] disconnected

Re: Talend Standard Job: Error while loading data into HDFS

manodwhb — Thu, 10 Sep 2020 09:18:42 GMT

@Rashmi Bector , check the below link.

https://help.talend.com/reader/WngYltFyZMz6FwUht_7S0A/Rh0fJUCV3~zDgwPC1gXfHQ

Thanks,

Manohar

topic Re: Talend Standard Job: Error while loading data into HDFS in Talend Studio

Talend Standard Job: Error while loading data into HDFS

Re: Talend Standard Job: Error while loading data into HDFS