sqoop import error parquet

Anonymous — Sat, 16 Nov 2024 06:25:34 GMT

Talend Studio Data Fabric vs 7.1.1

CDH: 5.16.1

Cloudera Manager: 5.16.1

Kerberos : True

TLS/SSL : Tue

sqoop import from oracle to cloudera compression parquet file :

error log :

[DEBUG]: org.apache.sqoop.mapreduce.DataDrivenImportJob - Using table class: QueryResult
Exception in thread "main" java.lang.NoClassDefFoundError: org/kitesdk/data/mapreduce/DatasetKeyOutputFormat
at org.apache.sqoop.mapreduce.DataDrivenImportJob.getOutputFormatClass(DataDrivenImportJob.java:211)
at org.apache.sqoop.mapreduce.ImportJobBase.configureOutputFormat(ImportJobBase.java:101)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:266)
at org.apache.sqoop.manager.SqlManager.importQuery(SqlManager.java:747)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:515)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:621)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at project_talend_lab_egl.test_sqoop_import_parquet_success_copy_0_4.test_sqoop_import_parquet_success_Copy.tSqoopImport_1Process(test_sqoop_import_parquet_success_Copy.java:1071)
at project_talend_lab_egl.test_sqoop_import_parquet_success_copy_0_4.test_sqoop_import_parquet_success_Copy.runJobInTOS(test_sqoop_import_parquet_success_Copy.java:2590)
at project_talend_lab_egl.test_sqoop_import_parquet_success_copy_0_4.test_sqoop_import_parquet_success_Copy.main(test_sqoop_import_parquet_success_Copy.java:2162)
Caused by: java.lang.ClassNotFoundException: org.kitesdk.data.mapreduce.DatasetKeyOutputFormat
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

models installed on Taledend Studio 7.1.1 :

if I add TLibraryLoad the in job and add th jar by hand. The job works without problems :

load jar :

kite-data-mapreduce-1.0.0-cdh5.12.1.jar - job OK
kite-data-mapreduce-1.0.0-cdh5.16.1.jar - job OK

screenshot :

job test_sqoop_import_parquet_success_Copy ended at 17:16 05/03/2019. [exit code=0]

in old version talend studio Data Fabric 6.5.1, CDH 5.14.x, kerberos and ssl enabled on cloudera, the job sqoop import is not necessary add tLibraryLoad for add the jar by hand .

Is this a problem with new versions?
Is there any documentation?
Is it possible to update the documentation if it does not exist?

Thx,

Re: sqoop import error parquet

Anonymous — Wed, 03 Apr 2019 11:21:58 GMT

Hello,

Thanks for letting us know this issue. We need more information to log it as a bug.

From talend bug tracker, there is no a related issue in V 7.1.1.

Does this issue repro when you use CDH 5.14.x in V 7.1.1? Are you getting some error message when install additional talend packages?

Best regards

Sabrina

Re: sqoop import error parquet

Anonymous — Fri, 24 May 2019 10:45:21 GMT

Hi,

the my responsed in red:

Does this issue repro when you use CDH 5.14.x in V 7.1.1?

yes confirmed error CDH 5.14.x and Talend 7.1.1

Are you getting some error message when install additional talend packages?

________________________________

the only error is “Class not found …” .

topic Re: sqoop import error parquet in Talend Studio

sqoop import error parquet

Re: sqoop import error parquet

Re: sqoop import error parquet