topic Re: tPigload failed with HCatLoader Load function in Talend Studio

tPigload failed with HCatLoader Load function

Anonymous — Sat, 16 Nov 2024 11:37:06 GMT

Hi,
tPigload fails to connect to Hive table with HCatLoader Load function, but retrieving the table via Pig console succeeds. Appreciate your help in regards to how to troubleshoot the issue.
Version of Talend Open Studio: 5.5.0
Hadoop: HDP 2.1 for Windows
1. retrieve table from Pig console - succeeded
a = LOAD 'Talend.weblog' using org.apache.hcatalog.pig.HCatLoader();
2. Use tPigload & tPigStoreResult components - failed
Job log
=====
User: molin
Name: BIGDATADEMO_test_0.1_tPigLoad_1
Application Type: MAPREDUCE
Application Tags:
State: FAILED
FinalStatus: FAILED
Started: 10-Jun-2014 10:37:42
Elapsed: 4sec
Tracking URL: History Diagnostics: Application application_1402325418471_0006 failed 2 times due to AM Container for appattempt_1402325418471_0006_000002 exited with exitCode: 1 due to: Exception from container-launch: org.apache.hadoop.util.Shell$ExitC

To see the whole post, download it here

Re: tPigload failed with HCatLoader Load function

Anonymous — Fri, 27 Jun 2014 03:44:28 GMT

I find the error is exactly the same with as error described at http://www.srccodes.com/p/article/46/noclassdeffounderror-org-apache-hadoop-service-compositeservice-shell-exitcodeexception-classnotfoundexception
However, I added the CLASSPATH property in yarn-site.xml according to the advice of the post, the error still remains. could anyone shed me a light?
thanks,
Mo

Re: tPigload failed with HCatLoader Load function

Anonymous — Sat, 05 Jul 2014 08:56:05 GMT

I think I could find the cause.
After install trial Enterprise studio, I am now able to trace into the job code. I found that that Talend studio ignores the classpath setting in ?yarn-site.xml? but always adds below classpath as the properties in the function tPigLoad_1Process() , which is actually wrong classpath for HDP Windows.
props_tPigLoad_1.put("yarn.application.classpath", "/etc/hadoop/conf,/usr/lib/hadoop/*,/usr/lib/hadoop/lib/*,/usr/lib/hadoop-hdfs/*,/usr/lib/hadoop-hdfs/lib/*,/usr/lib/hadoop-yarn/*,/usr/lib/hadoop-yarn/lib/*,/usr/lib/hadoop-mapreduce/*,/usr/lib/hadoop-mapreduce/lib/*");
The advice/help I need
1. Is this a bug for tPigload component? Where can I report this issue?
2. The debug trace stops at ?pigServer_tPigLoad_1.executeBatch();? which tell me that ?Source not found? for the file PigServer.calss. where can I download the source of PigServer.calss so that I can continue trace to locate the problematic call?
Thank you,
Mo

Re: tPigload failed with HCatLoader Load function

Anonymous — Mon, 07 Jul 2014 04:07:13 GMT

Hi Nemolin,
Could you please report a ticket on Talend Support Portal, so that our collgagues from support team will check if it is a bug with priority, through the support cycle.
Best regards
Sabrina