Error while running my big data application NPE

KarthikGs — Sat, 16 Nov 2024 10:06:57 GMT

Hi,
Here i'm basically running a small application which will fetch data from a hbase table, and apply some transformations on it and write the output as a parquet file in HDFS.

tHbaseInput >> tMap >> tFileOutputParquet

I can build the jar with out any errors but while running the job with a .sh script in the cluster, i'm facing these exceptions.


Running job: job_1472265299867_8850
 map 0% reduce 0%
Task Id : attempt_1472265299867_8850_m_000001_0, Status : FAILED
Error: java.lang.NullPointerException
        at parquet.io.api.Binary.fromString(Binary.java:211)
        at parquet.example.data.Group.append(Group.java:106)
        at local_project.karthik_test_1_0_1.Karthik_Test_1$TalendParquetOutputMapper_tFileOutputParquet_1_Mapper_1.map(Karthik_Test_1.java:15991)
        at local_project.karthik_test_1_0_1.Karthik_Test_1$TalendParquetOutputMapper_tFileOutputParquet_1_Mapper_1.map(Karthik_Test_1.java:1)
        at org.talend.hadoop.mapred.lib.Chain$ChainOutputCollector.collect(Chain.java:462)
        at local_project.karthik_test_1_0_1.Karthik_Test_1$tMap_1Mapper.map(Karthik_Test_1.java:15816)
        at local_project.karthik_test_1_0_1.Karthik_Test_1$tMap_1Mapper.map(Karthik_Test_1.java:1)
        at org.talend.hadoop.mapred.lib.ChainMapper.map(ChainMapper.java:63)
        at org.talend.hadoop.mapred.lib.DelegatingMapper.map(DelegatingMapper.java:44)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:435)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1566)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)

1. I have kept all the fields as nullable , but used this as a default value "" for string.
2.Which default values should i assign for the empty fields of other data types?? (date,Integer etc..) These might cause the NPE.
But the rest of the errors i cant understand. How to resolve these?
Please help me out.
Thanks in advance.

Re: Error while running my big data application NPE

Anonymous — Thu, 09 Feb 2017 01:50:53 GMT

Hi,
What's the build version you are using?
Could you please show us your job setting screenshots which will be helpful for us to address your issue.
Best regards
Sabrina

topic Error while running my big data application NPE in Talend Studio

Error while running my big data application NPE

Re: Error while running my big data application NPE