Hi all,
I am trying to load a file from HDFS into a Hive table using tHiveRow component.
Talend BigData Studio 5.6
Hadoop Hortonworks 2.1 with High Availability feature
Under advanced settings for tHiveRow I have:
dfs.client.use.datanode.hostname true
dfs.datanode.use.datanode.hostname true
dfs.nameservices HDPCLUSTER
dfs.ha.namenodes.HDPCLUSTER no,nt
dfs.namenode.rpc-address.HDPCLUSTER.no hdp:8020
dfs.namenode.rpc-address.HDPCLUSTER.nt hdp1:8020
dfs.client.failover.proxy.provider.HDPCLUSTER org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
When I run the job it's throwin the following exception:
FAILED: RuntimeException java.io.IOException: Couldn't create proxy provider class org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider java.lang.RuntimeException: java.io.IOException: Couldn't create proxy provider class org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider at org.apache.hadoop.hive.ql.Context.getScratchDir(Context.java:222) at org.apache.hadoop.hive.ql.Context.getExternalScratchDir(Context.java:278) at org.apache.hadoop.hive.ql.Context.getExternalTmpPath(Context.java:344) at org.apache.hadoop.hive.ql.parse.LoadSemanticAnalyzer.analyzeInternal(LoadSemanticAnalyzer.java:246) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:426) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:322) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:976) at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:969) at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:99) at org.apache.hive.service.cli.operation.SQLOperation.run(SQLOperation.java:172) at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:231) at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:218) at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:233) at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:346) at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:230) at data_integration.hiveexample_0_1.HiveExample.tHiveRow_1Process(HiveExample.java:549) at data_integration.hiveexample_0_1.HiveExample.tLibraryLoad_1Process(HiveExample.java:383) at data_integration.hiveexample_0_1.HiveExample.runJobInTOS(HiveExample.java:801) at data_integration.hiveexample_0_1.HiveExample.main(HiveExample.java:682) Caused by: java.io.IOException: Couldn't create proxy provider class org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider at org.apache.hadoop.hdfs.NameNodeProxies.createFailoverProxyProvider(NameNodeProxies.java:456) at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:148) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:579) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:524) at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:146) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2397) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:89) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2431) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2413) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:368) at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296) at org.apache.hadoop.hive.ql.Context.getScratchDir(Context.java:210) ... 19 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.hadoop.hdfs.NameNodeProxies.createFailoverProxyProvider(NameNodeProxies.java:445) ... 30 more Caused by: java.lang.RuntimeException: Could not find any configured addresses for URI hdfs://HDPCLUSTER/tmp/hive-hive/hive_2015-01-16_12-53-36_655_5380462070013163379-1 at org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider.<init>(ConfiguredFailoverProxyProvider.java:93) ... 35 more Any ideas why?
Thank you
