topic Re: tPigLoad cannot load any file from HDFS, it is getting the null value in Talend Studio

tPigLoad cannot load any file from HDFS, it is getting the null value

Anonymous — Sat, 16 Nov 2024 11:06:18 GMT

Hi, Guys
I am using your talend big data open studio to testing for big data, and running into an issue
When I tried to use tPigLoad to load file in HDFS, it gives the error please see the screen shot.
I have tried every solution in google, but none does work.

this is my testing schema in hdfs

please help me. Thank you!

I don't know why the screen shot is missed
please see the error
Starting job T_Pigload_Test at 03:08 10/08/2015.
connecting to socket on port 3587
connected
connecting to socket on port 4700
connected
: org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
objc: Class JavaLaunchHelper is implemented in both /Library/Java/JavaVirtualMachines/jdk1.8.0_11.jdk/Contents/Home/bin/java and /Library/Java/JavaVirtualMachines/jdk1.8.0_11.jdk/Contents/Home/jre/lib/libinstrument.dylib. One of the two will be used. Which one is undefined.
: org.apache.hadoop.mapreduce.JobSubmitter - Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
: org.apache.hadoop.mapreduce.JobSubmitter - No job jar file set. User classes may not be found. See Job or Job#setJar(String).
: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure.
: org.apache.pig.tools.pigstats.mapreduce.MRPigStatsUtil - 1 map reduce job(s) failed!
disconnected
disconnected
Job T_Pigload_Test ended at 03:09 10/08/2015.

Re: tPigLoad cannot load any file from HDFS, it is getting the null value

Anonymous — Mon, 10 Aug 2015 16:53:55 GMT

FYI, in the document from talend open big studio, I can successfully run the first 3 example
A_HCatalog_Create
B_HCatalog_Load
C_HCatalog_Read
I have no issue about connect to HCatalog, HDFS and namenode. I setup my own vm cluster and can run the pig command inside the pig shell (grunt) in the cluster.
Just cannot trigger the tPigLoad component in talend to the cluster, I believe the connection to namenode from talend to hadoop cluster is good. Can someone help me?
Thank you so much.

Re: tPigLoad cannot load any file from HDFS, it is getting the null value

_AnonymousUser — Wed, 14 Oct 2015 12:01:29 GMT

I'm having the same issue - same stack trace.
I don't seem able to post screenshots, but my flow is essentially:
tRowGenerator -> tPigLoad --> tPigCode --> tPigStoreResult
If I disable the StoreResult component, I get no errors...

Re: tPigLoad cannot load any file from HDFS, it is getting the null value

Anonymous — Tue, 26 Jan 2016 20:59:51 GMT

Hi
Were you able to get this problem resolved? I am facing the same issue. Please let me know how you resolved the issue
Thanks
Krithika

Re: tPigLoad cannot load any file from HDFS, it is getting the null value

Anonymous — Thu, 28 Jan 2016 07:26:57 GMT

Hi Krithika,
On which build version you got this issue? What's your work flow?
Best regards
Sabrina

Re: tPigLoad cannot load any file from HDFS, it is getting the null value

Anonymous — Thu, 28 Jan 2016 13:23:19 GMT

Talend version 6.1.1
TPigload- tpigstoreresult
Reading a file from hdfs in mapreduce mode and storing the results in a different directory in hdfs
Can you please help? It is urgentt

Re: tPigLoad cannot load any file from HDFS, it is getting the null value

Anonymous — Fri, 29 Jan 2016 17:12:14 GMT

Can anyone pls respond to the questions above?

Re: tPigLoad cannot load any file from HDFS, it is getting the null value

Anonymous — Mon, 01 Feb 2016 06:32:27 GMT

Hi kswamina,
Are you able to connect to HDFS? What's your OS and hadoop version?
Is your Talend Studio Java version is the same as the cluster? 1.8? More information will be preferred.
Best regards
Sabrina

Re: tPigLoad cannot load any file from HDFS, it is getting the null value

Anonymous — Mon, 01 Feb 2016 15:13:19 GMT

Hi Sabrina,
Yes, I am able to connect to HDFS. I am using windows 7 enterprise. Hadoop version is 2.7.1. Java version 1.8.0_65 in both talend and cluster.

Namenode URI "hdfs://10.56.70.93:9000"
Resource Manager "10.56.70.93:8032"
Resource Manager Scheduler Address: "10.56.70.93:8030"
If I remove the tPigStoreResult component from the job, i don't get any error.
Please helpp. If you need more information, please let me know
thanks
Krithika

Re: tPigLoad cannot load any file from HDFS, it is getting the null value

Anonymous — Mon, 01 Feb 2016 15:15:33 GMT

Hi Sabrina,
Also, please find attached the screenshot for the above issue

Re: tPigLoad cannot load any file from HDFS, it is getting the null value

Anonymous — Tue, 02 Feb 2016 01:32:12 GMT

Hi,
When i export the job and run it as shell script, this is the error message i get.
6/02/01 20:25:13 INFO mapReduceLayer.MapReduceLauncher: job job_1454362501785_0008 has failed! Stop running all dependent jobs
6/02/01 20:25:13 INFO mapReduceLayer.MapReduceLauncher: 100% complete
6/02/01 20:25:13 INFO client.RMProxy: Connecting to ResourceManager at /10.56.70.93:8032
6/02/01 20:25:13 INFO client.RMProxy: Connecting to ResourceManager at /10.56.70.93:8032
6/02/01 20:25:13 ERROR mapreduce.MRPigStatsUtil: 1 map reduce job(s) failed!
6/02/01 20:25:13 INFO mapreduce.SimplePigStats: Script Statistics:
HadoopVersionPigVersionUserIdStartedAtFinishedAtFeatures
2.7.1.2.3.2.0-29500.15.0.2.3.2.0-2950skrithika2016-02-01 20:25:072016-02-01 20:25:13UNKNOWN
Failed!
Failed Jobs:
JobIdAliasFeatureMessageOutputs
job_1454362501785_0008tPigLoad_1_row1_RESULTMAP_ONLYMessage: Job failed!/user/skrithika/pigstore,
Input(s):
Failed to read data from "/user/skrithika/test_data/data.txt"
Output(s):
Failed to produce result in "/user/skrithika/pigstore"
Counters:
Total records written : 0
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0
Job DAG:
job_1454362501785_0008

16/02/01 20:25:13 INFO mapReduceLayer.MapReduceLauncher: Failed!

Re: tPigLoad cannot load any file from HDFS, it is getting the null value

Anonymous — Tue, 02 Feb 2016 06:44:37 GMT

Hi,
Have you already checked document: TalendHelpCenterupported Hadoop distribution versions to see if you are on supported Hadoop distribution?
Best regards
Sabrina

Re: tPigLoad cannot load any file from HDFS, it is getting the null value

Anonymous — Tue, 02 Feb 2016 12:13:20 GMT

Yes, it is supported. Checked. This could be a yarn issue?

Re: tPigLoad cannot load any file from HDFS, it is getting the null value

Anonymous — Fri, 11 Mar 2016 01:26:52 GMT

it looks more like a code generation issue.