tPigLoad, tPigJoin, tPigFilterColumns et tPigStoreResult

Anonymous — Sat, 16 Nov 2024 12:05:05 GMT

hello everyone,
i tried this tuto:
https://help.talend.com/search/all?query=tPigJoin&content-lang=en
i searched on the net how to fix the error but i didn't find how.
any help please!
Thank youuu
________________________________________________________________
3/03/13 14:44:50 INFO executionengine.HExecutionEngine: Connecting to hadoop file system at: file:///
13/03/13 14:44:51 INFO pigstats.ScriptState: Pig features used in the script: HASH_JOIN
13/03/13 14:44:51 INFO rules.ColumnPruneVisitor: Columns pruned for tPigJoin_1_LOOKUP: $1, $2, $3, $4
13/03/13 14:44:51 INFO mapReduceLayer.MRCompiler: File concatenation threshold: 100 optimistic? false
13/03/13 14:44:51 INFO mapReduceLayer.MRCompiler$LastInputStreamingOptimizer: Rewrite: POPackage->POForEach to POJoinPackage
13/03/13 14:44:51 INFO mapReduceLayer.MultiQueryOptimizer: MR plan size before optimization: 1
13/03/13 14:44:51 INFO mapReduceLayer.MultiQueryOptimizer: MR plan size after optimization: 1
13/03/13 14:44:51 INFO pigstats.ScriptState: Pig script settings are added to the job
13/03/13 14:44:51 INFO mapReduceLayer.JobControlCompiler: mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
13/03/13 14:44:51 INFO mapReduceLayer.JobControlCompiler: Setting up single store job
13/03/13 14:44:51 INFO mapReduceLayer.JobControlCompiler: BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=0
13/03/13 14:44:51 INFO mapReduceLayer.JobControlCompiler: Neither PARALLEL nor default parallelism is set for this job. Setting number of reducers to 1
13/03/13 14:44:51 INFO mapReduceLayer.MapReduceLauncher: 1 map-reduce job(s) waiting for submission.
13/03/13 14:44:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
13/03/13 14:44:51 ERROR security.UserGroupInformation: PriviledgedActionException as:admin cause:java.io.IOException: Failed to set permissions of path: \tmp\hadoop-admin\mapred\staging\admin-1787914753\.staging to 0700
13/03/13 14:44:51 INFO mapReduceLayer.MapReduceLauncher: 0% complete
13/03/13 14:44:51 INFO mapReduceLayer.MapReduceLauncher: job null has failed! Stop running all dependent jobs
13/03/13 14:44:51 INFO mapReduceLayer.MapReduceLauncher: 100% complete
13/03/13 14:44:51 WARN mapReduceLayer.Launcher: There is no log file to write to.
13/03/13 14:44:51 ERROR mapReduceLayer.Launcher: Backend error message during job submission
java.io.IOException: Failed to set permissions of path: \tmp\hadoop-admin\mapred\staging\admin-1787914753\.staging to 0700
at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:682)
at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:655)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:856)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:824)
at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
at org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
at org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279)
at java.lang.Thread.run(Unknown Source)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:260)
13/03/13 14:44:52 ERROR pigstats.SimplePigStats: ERROR 2997: Unable to recreate exception from backend error: java.io.IOException: Failed to set permissions of path: \tmp\hadoop-admin\mapred\staging\admin-1787914753\.staging to 0700
at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:682)
at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:655)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:856)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:824)
at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
at org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
at org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279)
at java.lang.Thread.run(Unknown Source)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:260)
13/03/13 14:44:52 ERROR pigstats.PigStatsUtil: 1 map reduce job(s) failed!
13/03/13 14:44:52 INFO pigstats.SimplePigStats: Detected Local mode. Stats reported below may be incomplete
13/03/13 14:44:52 INFO pigstats.SimplePigStats: Script Statistics:
HadoopVersion PigVersion UserId StartedAt FinishedAt Features
1.0.0 0.10.0 admin 2013-03-13 14:44:51 2013-03-13 14:44:52 HASH_JOIN
Failed!
Failed Jobs:
JobId Alias Feature Message Outputs
N/A tPigFilterColumns_1_RESULT,tPigJoin_1_JOIN_RESULT,tPigJoin_1_LOOKUP,tPigJoin_1_RESULT,tPigLoad_1_RESULT HASH_JOIN Message: java.io.IOException: Failed to set permissions of path: \tmp\hadoop-admin\mapred\staging\admin-1787914753\.staging to 0700
at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:682)
at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:655)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:856)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:824)
at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
at org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
at org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279)
at java.lang.Thread.run(Unknown Source)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:260)
C:/Hadoop/exemple/outputFile.csv,
Input(s):
Failed to read data from "C:/Hadoop/exemple/refFile.csv"
Failed to read data from "C:/Hadoop/exemple/inputFile.csv"
Output(s):
Failed to produce result in "C:/Hadoop/exemple/outputFile.csv"
Job DAG:
null

13/03/13 14:44:52 INFO mapReduceLayer.MapReduceLauncher: Failed!
_________________________________________________________

Re: tPigLoad, tPigJoin, tPigFilterColumns et tPigStoreResult

Anonymous — Sun, 17 Mar 2013 19:52:14 GMT

I'm trying to run it with the "Local Mode"...
any answers please?

Re: tPigLoad, tPigJoin, tPigFilterColumns et tPigStoreResult

Anonymous — Tue, 19 Mar 2013 23:33:20 GMT

Hi
I don't use Hadoop on windows. Perhaps http://stackoverflow.com/questions/10509427/hadoop-in-windows might offer some insights.

Ciaran

topic tPigLoad, tPigJoin, tPigFilterColumns et tPigStoreResult in Talend Studio

tPigLoad, tPigJoin, tPigFilterColumns et tPigStoreResult

Re: tPigLoad, tPigJoin, tPigFilterColumns et tPigStoreResult

Re: tPigLoad, tPigJoin, tPigFilterColumns et tPigStoreResult