topic Re: Error in HDFSOutput outside Open Studio in Linux/cron in Talend Studio

Error in HDFSOutput outside Open Studio in Linux/cron

SJ3 — Sat, 16 Nov 2024 08:05:20 GMT

Hi,

I am pulling some log files and putting it under a Hadoop folder. I can run this job in the big data open studio but when I try to run it in cron/linux, it throws following error:

Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.conf.Configuration.addDeprecations([Lorg/apache/hadoop/conf/Configuration$DeprecationDeltaV
at org.apache.hadoop.hdfs.HdfsConfiguration.addDeprecatedKeys(HdfsConfiguration.java:66)

I thought may be this was because of different Hadoop versions so I checked I have following HDFS version:

HDFS

2.7.1.2.4

then I tried to change the versions of Hadoop under HDFSOutput component

But I couldn't find the same version. Now I am not sure how I can make this job work on linux/cron. Any suggestions?

Thanks!

Re: Error in HDFSOutput outside Open Studio in Linux/cron

SJ3 — Mon, 18 Jun 2018 17:38:56 GMT

I guess no one had this issue before

Re: Error in HDFSOutput outside Open Studio in Linux/cron

Anonymous — Tue, 19 Jun 2018 08:39:06 GMT

Hello,

The bigdata job works perfectly fine in studio but not in cmd(.sh file)? Are you using tHDFSconnection and tHBASEconnection in the same Job? Could you please indicate on which talend build version you got this issue?

Best regards

Sabrina

Re: Error in HDFSOutput outside Open Studio in Linux/cron

SJ3 — Tue, 19 Jun 2018 17:22:37 GMT

I am using version 6.3.1. And I was able to solve this problem by not using any context variable under HDFS components. I have also used same HDFS connections

in tHDFSOutput and tHDFSList, which I have created under Hadoop Cluster - Metadata options. Thanks for your help though Sabrina!