Spark Connection Error

[nnmh]

I have an error while trying to use spark Batching in connecting spark with hive and hadoop

The error :

org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.thrift.TApplicationException: Invalid method name: 'get_database_req'

at org.apache.hadoop.hive.ql.metadata.Hive.getDatabase(Hive.java:2137) ~[hive-exec-3.1.3000.7.1.1.0-565.jar:3.1.3000.7.1.1.0-565]

at org.apache.hadoop.hive.ql.metadata.Hive.databaseExists(Hive.java:2122) ~[hive-exec-3.1.3000.7.1.1.0-565.jar:3.1.3000.7.1.1.0-565]

at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$databaseExists$1.apply$mcZ$sp(HiveClientImpl.scala:415) ~[spark-hive_2.11-2.4.0.7.1.1.0-565.jar:2.4.0.7.1.1.0-565]

at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$databaseExists$1.apply(HiveClientImpl.scala:415) ~[spark-hive_2.11-2.4.0.7.1.1.0-565.jar:2.4.0.7.1.1.0-565]

at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$databaseExists$1.apply(HiveClientImpl.scala:415) ~[spark-hive_2.11-2.4.0.7.1.1.0-565.jar:2.4.0.7.1.1.0-565]

at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$withHiveState$1.apply(HiveClientImpl.scala:341) ~[spark-hive_2.11-2.4.0.7.1.1.0-565.jar:2.4.0.7.1.1.0-565]

at org.apache.spark.sql.hive.client.HiveClientImpl.liftedTree1$1(HiveClientImpl.scala:271) [spark-hive_2.11-2.4.0.7.1.1.0-565.jar:2.4.0.7.1.1.0-565]

at org.apache.spark.sql.hive.client.HiveClientImpl.retryLocked(HiveClientImpl.scala:270) [spark-hive_2.11-2.4.0.7.1.1.0-565.jar:2.4.0.7.1.1.0-565]

at org.apache.spark.sql.hive.client.HiveClientImpl.withHiveState(HiveClientImpl.scala:324) [spark-hive_2.11-2.4.0.7.1.1.0-565.jar:2.4.0.7.1.1.0-565]

at org.apache.spark.sql.hive.client.HiveClientImpl.databaseExists(HiveClientImpl.scala:414) [spark-hive_2.11-2.4.0.7.1.1.0-565.jar:2.4.0.7.1.1.0-565]

at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:219) [spark-hive_2.11-2.4.0.7.1.1.0-565.jar:2.4.0.7.1.1.0-565]

at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:219) [spark-hive_2.11-2.4.0.7.1.1.0-565.jar:2.4.0.7.1.1.0-565]

at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:219) [spark-hive_2.11-2.4.0.7.1.1.0-565.jar:2.4.0.7.1.1.0-565]

at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97) [spark-hive_2.11-2.4.0.7.1.1.0-565.jar:2.4.0.7.1.1.0-565]

at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:218) [spark-hive_2.11-2.4.0.7.1.1.0-565.jar:2.4.0.7.1.1.0-565]

at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:136) [spark-sql_2.11-2.4.0.7.1.1.0-565.jar:2.4.0.7.1.1.0-565]

at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:124) [spark-sql_2.11-2.4.0.7.1.1.0-565.jar:2.4.0.7.1.1.0-565]

at org.apache.spark.sql.hive.HiveSessionStateBuilder.org$apache$spark$sql$hive$HiveSessionStateBuilder$$externalCatalog(HiveSessionStateBuilder.scala:39) [spark-hive_2.11-2.4.0.7.1.1.0-565.jar:2.4.0.7.1.1.0-565]

any Help Please

[Anonymous]

Hello,

Could you please clarify in which Talend version/edition you are? What's your connection mode? Screenshots of hive and hadoop connection setting will be preferred.

Best regards

Sabrina