topic Re: Talend Spark Job issue with Hive in Talend Studio

Talend Spark Job issue with Hive

msdawood — Sat, 16 Nov 2024 03:50:08 GMT

Hi Team,

I'm using Talend Data Fabric 7.1.2 and I'm trying to read Avro file in HDFS and load into Hive table, I'm getting below error (Attached detailed Error)

Caused by: java.lang.ClassNotFoundException: org.apache.tez.dag.api.SessionNotRunning

Thanks,

Mohammed

Re: Talend Spark Job issue with Hive

Anonymous — Sat, 14 Dec 2019 04:11:43 GMT

Hi,

Could you please double check whether you have installed all the dependent jar files for the Hive and HDFS components? Usually Studio will prompt you to install them.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

Re: Talend Spark Job issue with Hive

msdawood — Wed, 18 Dec 2019 00:56:31 GMT

Yes, we have crossed checked ,all the depended jars are installed...

Re: Talend Spark Job issue with Hive

Anonymous — Wed, 18 Dec 2019 02:42:01 GMT

Hi,

Could you please advise which jar you are trying to load using tLibraryLoad component? If all modules are loaded, I believe you do not need the tLibraryLoad component in your job.

In worst case, if you are trying to load the libraries manually, could you please check whether you have added import command in the advance section of tLibraryLoad? Lets also check opinion of @rhall and @xdshi

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

Re: Talend Spark Job issue with Hive

Anonymous — Wed, 18 Dec 2019 03:47:12 GMT

Hello,

From error message, it seems that there is configuration problem for the Tez jars. Could you please precise your environment? What's hadoop version are you using? Are you using Tez as engine for Hive instead of MR?

Best regards

Sabrina

Re: Talend Spark Job issue with Hive

msdawood — Fri, 20 Dec 2019 09:03:11 GMT

Yes @xdshi We are using Tez as engine for Hive

We are using EMR 5.15.0(Hadoop 2.8.3)

@nthampi We are importing jersey-client-1.9.jar