Getting error while executing any Map Reduce Jobs from Talend.

rkaruppiah — Sat, 16 Nov 2024 11:01:51 GMT

Hi,
I ‘m trying to execute a simple Map Reduce Job from Talend and getting error as below. I verified and nothing wrong in Talend Map Reduce Job design and Configurations settings. I just want to ensure that there is no issues regarding User privileges or other issues at server(Hadoop/or Talend) end. Can you please check if you assist on this?
tHDFSInput_1 ----> tFileOutputParquet
Below is the stack trace of the job execution log:
Starting job testHDFSinPut at 18:53 24/09/2015.
connecting to socket on port 3625
connected
15/09/24 18:53:52 INFO Configuration.deprecation: mapred.compress.map.output is deprecated. Instead, use mapreduce.map.output.compress
15/09/24 18:53:52 INFO Configuration.deprecation: mapred.map.output.compression.codec is deprecated. Instead, use mapreduce.map.output.compress.codec
15/09/24 18:53:54 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 460433 for rakarupp on ha-hdfs:nameservice1
15/09/24 18:53:54 INFO security.TokenCache: Got dt for hdfs://nameservice1; Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:nameservice1, Ident: (HDFS_DELEGATION_TOKEN token 460433 for rakarupp)
15/09/24 18:53:55 INFO Configuration.deprecation: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
15/09/24 18:53:55 INFO mapred.FileInputFormat: Total input paths to process : 1
15/09/24 18:53:55 INFO Configuration.deprecation: mapred.min.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize
15/09/24 18:53:55 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
15/09/24 18:53:56 INFO mapreduce.JobSubmitter: number of splits:2
15/09/24 18:53:56 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
15/09/24 18:53:56 INFO Configuration.deprecation: mapred.map.output.compression.codec is deprecated. Instead, use mapreduce.map.output.compress.codec
15/09/24 18:53:56 INFO Configuration.deprecation: mapred.compress.map.output is deprecated. Instead, use mapreduce.map.output.compress
15/09/24 18:53:56 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1443002529133_1078
15/09/24 18:53:56 INFO mapreduce.JobSubmitter: Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:nameservice1, Ident: (HDFS_DELEGATION_TOKEN token 460433 for rakarupp)
15/09/24 18:53:56 INFO impl.YarnClientImpl: Submitted application application_1443002529133_1078
15/09/24 18:53:56 INFO mapreduce.Job: The url to track the job:
15/09/24 18:53:56 INFO Configuration.deprecation: jobclient.output.filter is deprecated. Instead, use mapreduce.client.output.filter
Running job: job_1443002529133_1078
map 0% reduce 0%
causerg.apache.hadoop.security.AccessControlException: Client cannot authenticate via:
15/09/24 18:54:05 WARN ipc.Client: Exception encountered while connecting to the server : org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:
cause:java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:
15/09/24 18:56:14 INFO mapred.ClientServiceDelegate: Application state is completed. FinalApplicationStatus=FAILED. Redirecting to job history server
Task Id : attempt_1443002529133_1078_m_000000_0, Status : FAILED
Error: Java heap space
Error: Java heap space
Task Id : attempt_1443002529133_1078_m_000000_1, Status : FAILED
Task Id : attempt_1443002529133_1078_m_000001_0, Status : FAILED
Error: Java heap space
Error: Java heap space
Task Id : attempt_1443002529133_1078_m_000001_1, Status : FAILED
Task Id : attempt_1443002529133_1078_m_000000_2, Status : FAILED
Error: Java heap space
Error: Java heap space
Error: Java heap space
Task Id : attempt_1443002529133_1078_m_000001_2, Status : FAILED
Task Id : attempt_1443002529133_1078_m_000001_3, Status : FAILED
map 0% reduce NaN%
Job complete: job_1443002529133_1078
Job Failed:
java.io.IOException: Job failed!
at org.talend.hadoop.mapred.lib.MRJobClient.runJob(MRJobClient.java:154)
at ref_etl.testhdfsinput_0_1.testHDFSinPut.runMRJob(testHDFSinPut.java:1207)
at ref_etl.testhdfsinput_0_1.testHDFSinPut.tHDFSInput_1Process(testHDFSinPut.java:1053)
at ref_etl.testhdfsinput_0_1.testHDFSinPut.run(testHDFSinPut.java:1175)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at ref_etl.testhdfsinput_0_1.testHDFSinPut.runJobInTOS(testHDFSinPut.java:1133)
at ref_etl.testhdfsinput_0_1.testHDFSinPut.main(testHDFSinPut.java:1118)
disconnected
Job testHDFSinPut ended at 18:56 24/09/2015.

Re: Getting error while executing any Map Reduce Jobs from Talend.

Anonymous — Tue, 13 Oct 2015 08:23:24 GMT

Hello,
Which version and edition of Talend are you using ?
Rémy.

Re: Getting error while executing any Map Reduce Jobs from Talend.

_AnonymousUser — Mon, 25 Apr 2016 10:49:07 GMT

Hello,
Your log reads "AccessControlException: Client cannot authenticate via:"
Could you check your cluster connection configuration / context variables ?

topic Re: Getting error while executing any Map Reduce Jobs from Talend. in Talend Studio

Getting error while executing any Map Reduce Jobs from Talend.

Re: Getting error while executing any Map Reduce Jobs from Talend.

Re: Getting error while executing any Map Reduce Jobs from Talend.