Connect Talend to Hive in Hortonworks Sandbox

Anonymous · ‎2015-11-30

HI all,
I am new in the forum, sorry if this post already exists, but looking around I could not find any solution it could help.
I am trying to connect talend to Hive. I am running Talend 6.1.0 on a virtual machine Ubuntu 14.04 and the sandbox is the Hortonworks Sandbox 2.3.0. Both machine are guested on a Windows machine and both are on bridge.
I was able to transfer file on the HDFS using talend.
I am using the tHiveConnection and I input the following parameteres
Host "192.168.1.3" (The IP of my sandbox) Port "10000"
Database "default"
Then in Hadoop properties I have:
Set Resource Manager: "sandbox.hortonworks.com:8032"
Set Namenode URI : "hdfs://sandbox.hortonworks.com:8020"
Set resourcemanager scheduler address: "sandbox.hortonworks.com:8030"
Set jobhistory address: "sandbox.hortonworks.com:10020"
Note that sandbox.hortonworks.com is the hostname for HW IP and yes is on the hosts file.
When I run however the following error arise:
---------------------------------------------------------------------------------------------------------------------------------
Starting job hive_connection at 17:01 30/11/2015.

connecting to socket on port 4031
connected
Exception in component tHiveConnection_1
org.apache.hive.service.cli.HiveSQLException: Error while processing statement: Cannot modify mapreduce.framework.name at runtime. It is not in list of params that are allowed to be modified at runtime
    at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:256)
    at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:242)
    at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:254)
    at hdfs.hive_connection_0_1.hive_connection.tHiveConnection_1Process(hive_connection.java:339)
    at hdfs.hive_connection_0_1.hive_connection.runJobInTOS(hive_connection.java:636)
    at hdfs.hive_connection_0_1.hive_connection.main(hive_connection.java:493)
Caused by: org.apache.hive.service.cli.HiveSQLException: Error while processing statement: Cannot modify mapreduce.framework.name at runtime. It is not in list of params that are allowed to be modified at runtime
    at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:315)
    at org.apache.hive.service.cli.operation.HiveCommandOperation.runInternal(HiveCommandOperation.java:108)
disconnected
    at org.apache.hive.service.cli.operation.Operation.run(Operation.java:257)
    at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:388)
    at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:375)
    at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:274)
    at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:486)
    at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313)
    at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298)
    at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
    at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
    at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
    at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Job hive_connection ended at 17:01 30/11/2015.
--------------------------------------------------------------------------------------------------------------------------------------------
Can anyone help?

Anonymous · ‎2016-02-08

Hi Jegan,

My Hive is also running under Tez. I found a Jira, which perhaps could help us. Had not yet the time to check it, but it looks promising:
https://jira.talendforge.org/browse/TBD-1504

Regards

_AnonymousUser · ‎2016-02-09

Hi cdrweanz
Thanks for your reply, i tried to replicate the same whatever the steps which they have mentioned. But does not work out. Kindly share the settings which you have done in your thiveConnection as a screenshot.
Regards,
Jegan

Anonymous · ‎2016-02-09

Hi Jegan,
I have the same problem. Didn't work. It's still the same message.
Regards

Anonymous · ‎2016-02-17

Hello all,

I found the solution. The problem is that if you select from the repository your cluster and keep all standard settings, Talend tries to connect via the data node. For any reasons it's not working. If you change from Repository to Stand Alone and set the following settings it works:

The advanced settings doesn't need to set as well:

Regards,

_AnonymousUser · ‎2016-03-03

Hi cdrweanz,
Thanks a lot, Its working fine now.

Anonymous · ‎2016-03-11

Hi all,
I've tried all things i've seen but still not working for me.
Please, can anyone Help me with this problem and thanks in advance.

Anonymous · ‎2016-03-14

As cdrweanz mentioned, the workaround is to switch to a built-in property type and uncheck all the Hadoop properties for the tHiveConnection component. See TBD-3260

Anonymous · ‎2017-03-07

Screenshots provided by cdrweanz worked for me

Anonymous · ‎2017-10-13

Text and screenshots here:

https://www.talendforge.org/forum/viewtopic.php?pid=178079#p178079

====================================================

Hello all,

I found the solution. The problem is that if you select from the repository your cluster and keep all standard settings, Talend tries to connect via the data node. For any reasons it's not working. If you change from Repository to Stand Alone and set the following settings it works:

The advanced settings doesn't need to set as well:

Regards,

Last edited by cdrweanz (2016-02-17 14:21:47)

====================================================

Sandbox

Talend Studio

v6.x