tpigLoad - Job running for long time

Anonymous — Sat, 16 Nov 2024 11:04:18 GMT

Hi - I am new to Talend and facing some challenges with tpigLoad.
I developed one job, just read a file, using tpigLoad, but its running for long time and not showing any result. Is it something wrong with configuration?
Starting job Landing_Load_v2 at 10:54 29/08/2015.
connecting to socket on port 4006
connected
: org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
: org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
: org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://<ip>:8020/
: org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
: org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
: org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
: org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
: org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
: org.apache.hadoop.conf.Configuration.deprecation - mapred.textoutputformat.separator is deprecated. Instead, use mapreduce.output.textoutputformat.separator
: org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN
: org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
: org.apache.pig.data.SchemaTupleBackend - Key was not set... will not generate code.
: org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=}
: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
: org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
: org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at /<ip>:8050
: org.apache.pig.tools.pigstats.mapreduce.MRScriptState - Pig script settings are added to the job
: org.apache.hadoop.conf.Configuration.deprecation - mapred.job.reduce.markreset.buffer.percent is deprecated. Instead, use mapreduce.reduce.markreset.buffer.percent
: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
: org.apache.hadoop.conf.Configuration.deprecation - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - This job cannot be converted run in-process
: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Added jar file:/C:/Users/z062913/Desktop/big%20data/Talend/TOS_BD-20150508_1414-V5.6.2/workspace/.Java/lib/datafu-1.2.0.jar to DistributedCache through /tmp/temp-1107527774/tmp-361404915/datafu-1.2.0.jar
: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Added jar file:/C:/Users/z062913/Desktop/big%20data/Talend/TOS_BD-20150508_1414-V5.6.2/workspace/.Java/lib/pig-0.14.0.2.2.0.0-2041-core-h2.jar to DistributedCache through /tmp/temp-1107527774/tmp105674471/pig-0.14.0.2.2.0.0-2041-core-h2.jar
: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Added jar file:/C:/Users/z062913/Desktop/big%20data/Talend/TOS_BD-20150508_1414-V5.6.2/workspace/.Java/lib/automaton-1.11-8.jar to DistributedCache through /tmp/temp-1107527774/tmp1941076374/automaton-1.11-8.jar
: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Added jar file:/C:/Users/z062913/Desktop/big%20data/Talend/TOS_BD-20150508_1414-V5.6.2/workspace/.Java/lib/antlr-runtime-3.4.jar to DistributedCache through /tmp/temp-1107527774/tmp1377012023/antlr-runtime-3.4.jar
: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Added jar file:/C:/Users/z062913/Desktop/big%20data/Talend/TOS_BD-20150508_1414-V5.6.2/workspace/.Java/lib/guava-11.0.2.jar to DistributedCache through /tmp/temp-1107527774/tmp-1975486051/guava-11.0.2.jar
: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Added jar file:/C:/Users/z062913/Desktop/big%20data/Talend/TOS_BD-20150508_1414-V5.6.2/workspace/.Java/lib/joda-time-2.3.jar to DistributedCache through /tmp/temp-1107527774/tmp1117462283/joda-time-2.3.jar
: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job
: org.apache.pig.data.SchemaTupleFrontend - Key is false, will not generate code.
: org.apache.pig.data.SchemaTupleFrontend - Starting process to move generated code to distributed cacche
: org.apache.pig.data.SchemaTupleFrontend - Setting key with classes to deserialize []
: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
: org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker.http.address is deprecated. Instead, use mapreduce.jobtracker.http.address
: org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at /<ip>:8050
: org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
: org.apache.hadoop.mapreduce.JobSubmitter - Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
: org.apache.hadoop.mapreduce.JobSubmitter - No job jar file set. User classes may not be found. See Job or Job#setJar(String).
: org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
: org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
: org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1
: org.apache.hadoop.mapreduce.JobSubmitter - number of splits:1
: org.apache.hadoop.mapreduce.JobSubmitter - Submitting tokens for job: job_1440819158223_0002
: org.apache.hadoop.mapred.YARNRunner - Job jar is not present. Not adding any jar to the list of resources.
: org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted application application_1440819158223_0002
: org.apache.hadoop.mapreduce.Job - The url to track the job:
: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_1440819158223_0002
: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases tPigLoad_1_row1_RESULT
: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: tPigLoad_1_row1_RESULT,tPigLoad_1_row1_RESULT C: R:
: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Running jobs are
Thanks,
Robin

Re: tpigLoad - Job running for long time

Anonymous — Sat, 29 Aug 2015 07:27:10 GMT

Below are few more details ..
Talend version - 5.6
using Sandbox_HDP_2.2_VMWare

Re: tpigLoad - Job running for long time

Anonymous — Mon, 31 Aug 2015 03:59:06 GMT

Hi,
How did you set your tpigLoad component? Have you already checked component reference with related scenario:TalendHelpCenter:tPigLoad?
Is there any error message printed on console?
Best regards
Sabrina

Re: tpigLoad - Job running for long time

_AnonymousUser — Thu, 13 Oct 2016 18:55:50 GMT

Is there any demo video for talend Pig components? If yes please provide the link or guide me to a place where I can download it.
Thanks
MUS

topic tpigLoad - Job running for long time in Talend Studio

tpigLoad - Job running for long time

Re: tpigLoad - Job running for long time

Re: tpigLoad - Job running for long time

Re: tpigLoad - Job running for long time