I keep getting errors similar to the following on a simple Hive job - this Null Pointer Exception looks like a bug (even if there is an incorrect setting it's not handling it properly). I've double checked the settings and they're the same as I use in my SQuirreL JDBC client which works... this is connecting to a Hortonworks cluster via HiveServer2:
Starting job Hive_Test at 15:31 18/08/2014. connecting to socket on port 3508 connected : org.apache.hadoop.util.Shell - Failed to locate the winutils binary in the hadoop binary path java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries. at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:318) at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:333) at org.apache.hadoop.util.Shell.<clinit>(Shell.java:326) at org.apache.hadoop.hive.conf.HiveConf$ConfVars.findHadoopBinary(HiveConf.java:1158) at org.apache.hadoop.hive.conf.HiveConf$ConfVars.<clinit>(HiveConf.java:246) at org.apache.hadoop.hive.conf.HiveConf.<clinit>(HiveConf.java:81) at org.apache.hive.service.cli.thrift.EmbeddedThriftBinaryCLIService.<init>(EmbeddedThriftBinaryCLIService.java:35) at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:164) at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) at java.sql.DriverManager.getConnection(Unknown Source) at java.sql.DriverManager.getConnection(Unknown Source) at testproject.hive_test_0_1.Hive_Test.tHiveConnection_1Process(Hive_Test.java:371) at testproject.hive_test_0_1.Hive_Test.runJobInTOS(Hive_Test.java:1063) at testproject.hive_test_0_1.Hive_Test.main(Hive_Test.java:922) : org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable : hive.metastore - Trying to connect to metastore with URI thrift://hiveserver_host:10000 Exception in component tHiveConnection_1 java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:347) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:292) at org.apache.hive.service.cli.session.SessionManager.applyAuthorizationConfigPolicy(SessionManager.java:88) at org.apache.hive.service.cli.session.SessionManager.init(SessionManager.java:63) at org.apache.hive.service.CompositeService.init(CompositeService.java:59) at org.apache.hive.service.cli.CLIService.init(CLIService.java:110) at org.apache.hive.service.cli.thrift.EmbeddedThriftBinaryCLIService.<init>(EmbeddedThriftBinaryCLIService.java:35) at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:164) at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) at java.sql.DriverManager.getConnection(Unknown Source) at java.sql.DriverManager.getConnection(Unknown Source) at testproject.hive_test_0_1.Hive_Test.tHiveConnection_1Process(Hive_Test.java:371) at testproject.hive_test_0_1.Hive_Test.runJobInTOS(Hive_Test.java:1063) at testproject.hive_test_0_1.Hive_Test.main(Hive_Test.java:922) Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1413) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:62) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:72) at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2444) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2456) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:341) ... 13 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) at java.lang.reflect.Constructor.newInstance(Unknown Source) at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1411) ... 18 more Caused by: java.lang.NullPointerException at java.lang.ProcessBuilder.start(Unknown Source) at org.apache.hadoop.util.Shell.runCommand(Shell.java:445) at org.apache.hadoop.util.Shell.run(Shell.java:418) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650) at org.apache.hadoop.util.Shell.execCommand(Shell.java:739) at org.apache.hadoop.util.Shell.execCommand(Shell.java:722) at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:83) at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:52) at org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.getGroups(JniBasedUnixGroupsMappingWithFallback.java:50) at org.apache.hadoop.security.Groups.getGroups(Groups.java:139) at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1418) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:352) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:214) ... 23 more Exception in component tHiveInput_1 java.lang.NullPointerException at testproject.hive_test_0_1.Hive_Test.tHiveInput_1Process(Hive_Test.java:679) at testproject.hive_test_0_1.Hive_Test.runJobInTOS(Hive_Test.java:1075) at testproject.hive_test_0_1.Hive_Test.main(Hive_Test.java:922) disconnected Job Hive_Test ended at 15:31 18/08/2014.
I got past this today, I switched from embedded to standalone mode, although I'm positive I tried that yesterday as well as a few other things. Anyway, I'll mark this as resolved, although it's also worth noting that I've had a situation where job runs didn't start and had blank output after clicking Run, with both Run and Kill dialogs greyed out, only closing the whole app and reopening it got the job working again (without any changes to the job itself).
