Here is my case:
Talend Data Fabric 6.11
I created a job with 2 ELTHiveMap component, one's output as other's input,
I ran this job in Talend without any problem,
but after I build the job deploy to Cluster, start with the shell script cust_snapshot_acct_rltnp_master_run.sh
I got the error message:
...
: org.apache.hive.jdbc.HiveConnection - Error opening session
org.apache.thrift.TException: Unsupported Hive2 protocol
at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:461)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:178)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:233)
at project.cust_tsz_account_relationship_to_ez_cust_acct_rltnp_0_1.CUST_tsz_account_relationship_to_ez_cust_acct_rltnp.tELTHiveMap_1Process(CUST_tsz_account_relationship_to_ez_cust_acct_rltnp.java:684)
at project.cust_tsz_account_relationship_to_ez_cust_acct_rltnp_0_1.CUST_tsz_account_relationship_to_ez_cust_acct_rltnp.runJobInTOS(CUST_tsz_account_relationship_to_ez_cust_acct_rltnp.java:1173)
at project.cust_tsz_account_relationship_to_ez_cust_acct_rltnp_0_1.CUST_tsz_account_relationship_to_ez_cust_acct_rltnp.runJob(CUST_tsz_account_relationship_to_ez_cust_acct_rltnp.java:900)
at project.cust_snapshot_acct_rltnp_master_0_1.cust_snapshot_acct_rltnp_master.tRunJob_1Process(cust_snapshot_acct_rltnp_master.java:713)
at project.cust_snapshot_acct_rltnp_master_0_1.cust_snapshot_acct_rltnp_master.runJobInTOS(cust_snapshot_acct_rltnp_master.java:1124)
at project.cust_snapshot_acct_rltnp_master_0_1.cust_snapshot_acct_rltnp_master.main(cust_snapshot_acct_rltnp_master.java:841)
: project.cust_tsz_account_relationship_to_ez_cust_acct_rltnp_0_1.CUST_tsz_account_relationship_to_ez_cust_acct_rltnp - tELTHiveOutput_1 Could not establish connection to jdbc:hive2://xxxxxxx (I masked the connection information): Unsupported Hive2 protocol
Exception in component tELTHiveOutput_1
java.sql.SQLException: Could not establish connection to jdbc:hive2://xxxxxxx (I masked the connection information): Unsupported Hive2 protocol
at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:467)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:178)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:233)
at project.cust_tsz_account_relationship_to_ez_cust_acct_rltnp_0_1.CUST_tsz_account_relationship_to_ez_cust_acct_rltnp.tELTHiveMap_1Process(CUST_tsz_account_relationship_to_ez_cust_acct_rltnp.java:684)
at project.cust_tsz_account_relationship_to_ez_cust_acct_rltnp_0_1.CUST_tsz_account_relationship_to_ez_cust_acct_rltnp.runJobInTOS(CUST_tsz_account_relationship_to_ez_cust_acct_rltnp.java:1173)
at project.cust_tsz_account_relationship_to_ez_cust_acct_rltnp_0_1.CUST_tsz_account_relationship_to_ez_cust_acct_rltnp.runJob(CUST_tsz_account_relationship_to_ez_cust_acct_rltnp.java:900)
at project.cust_snapshot_acct_rltnp_master_0_1.cust_snapshot_acct_rltnp_master.tRunJob_1Process(cust_snapshot_acct_rltnp_master.java:713)
at project.cust_snapshot_acct_rltnp_master_0_1.cust_snapshot_acct_rltnp_master.runJobInTOS(cust_snapshot_acct_rltnp_master.java:1124)
at project.cust_snapshot_acct_rltnp_master_0_1.cust_snapshot_acct_rltnp_master.main(cust_snapshot_acct_rltnp_master.java:841)
Caused by: org.apache.thrift.TException: Unsupported Hive2 protocol
at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:461)
... 10 more
...
when I open the shell script cust_snapshot_acct_rltnp_master_run.sh
I found there are couple of hadoop related jar reference with different version:
#!/bin/sh
cd `dirname $0`
ROOT_PATH=`pwd`
java -Xms256M -Xmx1024M -cp .:$ROOT_PATH...:$ROOT_PATH/../lib/hadoop-hdfs-2.7.1.2.3.2.0-2950.jar...:$ROOT_PATH/../lib/hadoop-hdfs-2.6.0-amzn-0.jar...
We have the reference to hadoop-hdfs-2.7 and hadoop-hdfs-2.6 at same time, also with other jar like hive-jdbc have two version,
After I remove the lower version (because my cluster is 2.7), then the job work properly.
All other job without multiple ELTHiveMap component works find, I checked those job's shell script file, no duplicated versionof jar reference.
Can you please check what cause the problem?
really appreciated.
Thanks,
Hi,
For your
Talend Data Fabric
product, have you already reported a ticket on Talend Support Portal? In this way, we can give you a remote assistance through support cycle with priority.
Best regards
Sabrina