[resolved] Hive connector bug?

Anonymous — Sat, 16 Nov 2024 11:32:27 GMT

I keep getting errors similar to the following on a simple Hive job - this Null Pointer Exception looks like a bug (even if there is an incorrect setting it's not handling it properly). I've double checked the settings and they're the same as I use in my SQuirreL JDBC client which works... this is connecting to a Hortonworks cluster via HiveServer2:

Starting job Hive_Test at 15:31 18/08/2014.
 connecting to socket on port 3508
 connected
: org.apache.hadoop.util.Shell - Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
    at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:318)
    at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:333)
    at org.apache.hadoop.util.Shell.<clinit>(Shell.java:326)
    at org.apache.hadoop.hive.conf.HiveConf$ConfVars.findHadoopBinary(HiveConf.java:1158)
    at org.apache.hadoop.hive.conf.HiveConf$ConfVars.<clinit>(HiveConf.java:246)
    at org.apache.hadoop.hive.conf.HiveConf.<clinit>(HiveConf.java:81)
    at org.apache.hive.service.cli.thrift.EmbeddedThriftBinaryCLIService.<init>(EmbeddedThriftBinaryCLIService.java:35)
    at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:164)
    at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
    at java.sql.DriverManager.getConnection(Unknown Source)
    at java.sql.DriverManager.getConnection(Unknown Source)
    at testproject.hive_test_0_1.Hive_Test.tHiveConnection_1Process(Hive_Test.java:371)
    at testproject.hive_test_0_1.Hive_Test.runJobInTOS(Hive_Test.java:1063)
    at testproject.hive_test_0_1.Hive_Test.main(Hive_Test.java:922)
: org.apache.hadoop.util.NativeCodeLoader - Unable to load 
native-hadoop library for your platform... using builtin-java classes 
where applicable
: hive.metastore - Trying to connect to metastore with URI thrift://hiveserver_host:10000
Exception in component tHiveConnection_1
java.lang.RuntimeException:
 java.lang.RuntimeException: Unable to instantiate 
org.apache.hadoop.hive.metastore.HiveMetaStoreClient
    at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:347)
    at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:292)
    at org.apache.hive.service.cli.session.SessionManager.applyAuthorizationConfigPolicy(SessionManager.java:88)
    at org.apache.hive.service.cli.session.SessionManager.init(SessionManager.java:63)
    at org.apache.hive.service.CompositeService.init(CompositeService.java:59)
    at org.apache.hive.service.cli.CLIService.init(CLIService.java:110)
    at org.apache.hive.service.cli.thrift.EmbeddedThriftBinaryCLIService.<init>(EmbeddedThriftBinaryCLIService.java:35)
    at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:164)
    at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
    at java.sql.DriverManager.getConnection(Unknown Source)
    at java.sql.DriverManager.getConnection(Unknown Source)
    at testproject.hive_test_0_1.Hive_Test.tHiveConnection_1Process(Hive_Test.java:371)
    at testproject.hive_test_0_1.Hive_Test.runJobInTOS(Hive_Test.java:1063)
    at testproject.hive_test_0_1.Hive_Test.main(Hive_Test.java:922)
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
    at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1413)
    at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:62)
    at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:72)
    at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2444)
    at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2456)
    at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:341)
    ... 13 more
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
    at java.lang.reflect.Constructor.newInstance(Unknown Source)
    at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1411)
    ... 18 more
Caused by: java.lang.NullPointerException
    at java.lang.ProcessBuilder.start(Unknown Source)
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:445)
    at org.apache.hadoop.util.Shell.run(Shell.java:418)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:739)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:722)
    at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:83)
    at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:52)
    at org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.getGroups(JniBasedUnixGroupsMappingWithFallback.java:50)
    at org.apache.hadoop.security.Groups.getGroups(Groups.java:139)
    at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1418)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:352)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:214)
    ... 23 more
Exception in component tHiveInput_1
java.lang.NullPointerException
    at testproject.hive_test_0_1.Hive_Test.tHiveInput_1Process(Hive_Test.java:679)
    at testproject.hive_test_0_1.Hive_Test.runJobInTOS(Hive_Test.java:1075)
    at testproject.hive_test_0_1.Hive_Test.main(Hive_Test.java:922)
 disconnected
Job Hive_Test ended at 15:31 18/08/2014.

Re: [resolved] Hive connector bug?

Anonymous — Tue, 19 Aug 2014 17:35:51 GMT

I got past this today, I switched from embedded to standalone mode, although I'm positive I tried that yesterday as well as a few other things. Anyway, I'll mark this as resolved, although it's also worth noting that I've had a situation where job runs didn't start and had blank output after clicking Run, with both Run and Kill dialogs greyed out, only closing the whole app and reopening it got the job working again (without any changes to the job itself).

Re: [resolved] Hive connector bug?

Anonymous — Fri, 02 Sep 2016 07:45:19 GMT

you need to add win utils and the same in job properties and done

Re: [resolved] Hive connector bug?

Anonymous — Mon, 31 Oct 2016 06:50:55 GMT

Hi,
Here is a KB article about: TalendHelpCenter:The missing winutils.exe program in the Big Data Jobs.
Best regards
Sabrina

topic Re: [resolved] Hive connector bug? in Talend Studio

[resolved] Hive connector bug?

Re: [resolved] Hive connector bug?

Re: [resolved] Hive connector bug?

Re: [resolved] Hive connector bug?