Hi,
tPigload fails to connect to Hive table with HCatLoader Load function, but retrieving the table via Pig console succeeds. Appreciate your help in regards to how to troubleshoot the issue.
Version of Talend Open Studio: 5.5.0
Hadoop: HDP 2.1 for Windows
1. retrieve table from Pig console - succeeded
a = LOAD 'Talend.weblog' using org.apache.hcatalog.pig.HCatLoader();
2. Use tPigload & tPigStoreResult components - failed
Job log
=====
User: molin
Name: BIGDATADEMO_test_0.1_tPigLoad_1
Application Type: MAPREDUCE
Application Tags:
State: FAILED
FinalStatus: FAILED
Started: 10-Jun-2014 10:37:42
Elapsed: 4sec
Tracking URL: History Diagnostics: Application application_1402325418471_0006 failed 2 times due to AM Container for appattempt_1402325418471_0006_000002 exited with exitCode: 1 due to: Exception from container-launch: org.apache.hadoop.util.Shell$ExitC
I think I could find the cause.
After install trial Enterprise studio, I am now able to trace into the job code. I found that that Talend studio ignores the classpath setting in ?yarn-site.xml? but always adds below classpath as the properties in the function tPigLoad_1Process() , which is actually wrong classpath for HDP Windows.
props_tPigLoad_1.put("yarn.application.classpath", "/etc/hadoop/conf,/usr/lib/hadoop/*,/usr/lib/hadoop/lib/*,/usr/lib/hadoop-hdfs/*,/usr/lib/hadoop-hdfs/lib/*,/usr/lib/hadoop-yarn/*,/usr/lib/hadoop-yarn/lib/*,/usr/lib/hadoop-mapreduce/*,/usr/lib/hadoop-mapreduce/lib/*");
The advice/help I need
1. Is this a bug for tPigload component? Where can I report this issue?
2. The debug trace stops at ?pigServer_tPigLoad_1.executeBatch();? which tell me that ?Source not found? for the file PigServer.calss. where can I download the source of PigServer.calss so that I can continue trace to locate the problematic call?
Thank you,
Mo
Hi Nemolin,
Could you please report a ticket on
Talend Support Portal, so that our collgagues from support team will check if it is a bug with priority, through the support cycle.
Best regards
Sabrina