topic Hive: java.net.ConnectException: Connection refused in Talend Studio

Hive: java.net.ConnectException: Connection refused

Anonymous — Sat, 16 Nov 2024 10:53:12 GMT

Hi all,
I have a local installation of hadoop, hive, spark and so on and tried to use the Open Studio for BD (version: 6.1.1.) to process some example data using hive.
1. hadoop is running (its still on version 1.2.1)
2. hive metastore and hive is running
I can connect to hive using the "tHiveConnection" component and I can create and drop tables in hive using "tHiveRow" component but I CANNOT process data using the "tELTHiveOutput" component:

There is no map reduce job created, however I can process the data when I'm dircetly in the hive-shell.
According to my understanding, the main problem is:
ava.lang.RuntimeException: java.net.ConnectException: Call to localhost/127.0.0.1:8020 failed on connection exception: java.net.ConnectException: Connection refused
Any idea which parameters to set?!?!
Thanks for any hint.
Cheers

Re: Hive: java.net.ConnectException: Connection refused

Anonymous — Mon, 04 Jan 2016 12:13:48 GMT

Hello,
Which Hive mode do you use ? Standalone or embedded ?
Rémy.

Re: Hive: java.net.ConnectException: Connection refused

Anonymous — Mon, 04 Jan 2016 22:25:20 GMT

Hi,
I'm using the embedded mode.
My testenvironment ist a pseudo distributed cluster on my local machine.
Thanks for your reply.
Cheers
Gerd

Re: Hive: java.net.ConnectException: Connection refused

Anonymous — Wed, 06 Jan 2016 08:40:48 GMT

Hi,
Ok. Can you please make sure that you started the hive-metastore service on your local machine ? Also, did you start the hive-server2 service ? It would allow to try using the standalone mode which is actually recommanded.
Also, the error you have would mean that the namenode is not started. Can you access the namenode web page browsing http://127.0.0.1:50070 ?
Rémy.

Re: Hive: java.net.ConnectException: Connection refused

Anonymous — Wed, 06 Jan 2016 23:46:31 GMT

Hi,
I tried to check this already be myself:
1. metastore server is running: bin/hive --service metastore
Starting Hive Metastore Server
2. the hiveserver2 is started and running: bin/hive --service hiveserver2
3. this means I assume, that my hive installation is running, because I can work with hive using the console. An example in the hive console like SELECT COUNT(*) from tweets; creates a map reduce job like expected:

4. I can create a connection to my local hive installation and create the table with talend:

This is working good however, the connection mode is embedded:

Using the same connection(!!!), I create another hive table and try to fill it with some data using the tELTHiveOutput_1 component. which fails with the message:

Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks is set to 0 since there's no reduce operator
java.net.ConnectException: Call to gerdmac/192.168.2.101:50300 failed on connection exception: java.net.ConnectException: Connection refused

Namenode for sure is running:

I'm lost.
Cheers
Gerd

Re: Hive: java.net.ConnectException: Connection refused

Anonymous — Wed, 06 Jan 2016 23:51:58 GMT

Hi,
I tried to check this already be myself:
1. metastore server is running: bin/hive --service metastore
Starting Hive Metastore Server
2. the hiveserver2 is started and running: bin/hive --service hiveserver2
3. this means I assume, that my hive installation is running, because I can work with hive using the console. An example in the hive console like SELECT COUNT(*) from tweets; creates a map reduce job like expected:
4. I can create a connection to my local hive installation and create the table with talend:
This is working good however, the connection mode is embedded:
Using the same connection(!!!), I create another hive table and try to fill it with some data using the tELTHiveOutput_1 component. which fails with the message:

Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks is set to 0 since there's no reduce operator
java.net.ConnectException: Call to gerdmac/192.168.2.101:50300 failed on connection exception: java.net.ConnectException: Connection refused

Namenode for sure is running:
I'm lost.
Gerd