java.lang.NoClassDefFoundError when use tSqoopExport

fatahzx — Sat, 16 Nov 2024 00:53:35 GMT

I ran this job with talend big data platform version 7.2 and use java JDK version 8.0.202

i want to transfer file with bulk mode using sqoop from hdfs to sql server.

when use component tSqoopExport with java api option, job failed and show exception

Exception in thread "main" java.lang.NoClassDefFoundError: org/kitesdk/data/mapreduce/DatasetKeyInputFormat

at org.apache.sqoop.mapreduce.parquet.kite.KiteParquetExportJobConfigurator.getInputFormatClass(KiteParquetExportJobConfigurator.java:49)

my file fomat in hdfs is parquet. i think, that is the problem,.to read parquet file, sqoop need some jar file.

my question, how to solve this?or how to add jar in tsqoopexport component.

in full error like this:

[statistics] connecting to socket on port 4010

[statistics] connected

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".

SLF4J: Defaulting to no-operation (NOP) logger implementation

SLF4J: See http://www.slf4j.org/codes.html# StaticLoggerBinder for further details.

[WARN ]: org.apache.sqoop.ConnFactory - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.

Exception in thread "main" java.lang.NoClassDefFoundError: org/kitesdk/data/mapreduce/DatasetKeyInputFormat

at org.apache.sqoop.mapreduce.parquet.kite.KiteParquetExportJobConfigurator.getInputFormatClass(KiteParquetExportJobConfigurator.java:49)

at org.apache.sqoop.mapreduce.JdbcExportJob.getInputFormatClass(JdbcExportJob.java:126)

at org.apache.sqoop.mapreduce.JobBase.configureInputFormat(JobBase.java:302)

at org.apache.sqoop.mapreduce.ExportJobBase.configureInputFormat(ExportJobBase.java:241)

at org.apache.sqoop.mapreduce.JdbcExportJob.configureInputFormat(JdbcExportJob.java:73)

at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:433)

at org.apache.sqoop.manager.SQLServerManager.exportTable(SQLServerManager.java:170)

at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:93)

at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:112)

at org.apache.sqoop.Sqoop.run(Sqoop.java:146)

at pelayanan.bulk_insert_sql_0_1.BULK_INSERT_SQL.tSqoopExport_1Process(BULK_INSERT_SQL.java:584)

at pelayanan.bulk_insert_sql_0_1.BULK_INSERT_SQL.runJobInTOS(BULK_INSERT_SQL.java:1158)

at pelayanan.bulk_insert_sql_0_1.BULK_INSERT_SQL.main(BULK_INSERT_SQL.java:906)

Caused by: java.lang.ClassNotFoundException: org.kitesdk.data.mapreduce.DatasetKeyInputFormat

at java.net.URLClassLoader.findClass(URLClassLoader.java:382)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

... 13 more

[statistics] disconnected

Re: java.lang.NoClassDefFoundError when use tSqoopExport

Anonymous — Wed, 30 Dec 2020 05:04:03 GMT

Hello,

Could you please try to add tLibraryLoad component in your job and add relevant jar files by hand?

Best regards

Sabrina

Re: java.lang.NoClassDefFoundError when use tSqoopExport

fatahzx — Thu, 25 Feb 2021 18:29:31 GMT

still error. but i resolved with change format to textfile and add another component to convert to parquetfile

Re: java.lang.NoClassDefFoundError when use tSqoopExport

Anonymous — Fri, 26 Feb 2021 06:50:04 GMT

Hello,

Thanks for keep us updated to this issue and sharing your solution with us.

Best regards

Sabrina

topic Re: java.lang.NoClassDefFoundError when use tSqoopExport in Talend Studio

java.lang.NoClassDefFoundError when use tSqoopExport

Re: java.lang.NoClassDefFoundError when use tSqoopExport

Re: java.lang.NoClassDefFoundError when use tSqoopExport

Re: java.lang.NoClassDefFoundError when use tSqoopExport