Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Join us in NYC Sept 4th for Qlik's AI Reality Tour! Register Now
cancel
Showing results for 
Search instead for 
Did you mean: 
Eddie_Liu
Contributor
Contributor

Spark libjars can't be gotten

we have following job, this job will submit to spark yarn. JOB_CLS_SAP_SALES_HEAD is a big data batch. when we deploy the job, we found only 2 Spark libjars were got, actually it need more libjars, it made NoClassDefFoundError. what's wrong? could anyone help on it?

[INFO ]:xxxxxxxxxxxxxxxx.job_db_sap_sales_order_scheduling_0_1.JOB_DB_SAP_SALES_ORDER_SCHEDULING - tRunJob_2 - The child job 'digital_factory_phase_i.job_stg_sap_vbap_0_1.JOB_STG_SAP_VBAP' is done.

[INFO ]:xxxxxxxxxxxxxxxx.job_db_sap_sales_order_scheduling_0_1.JOB_DB_SAP_SALES_ORDER_SCHEDULING - tRunJob_3 - The child job 'JOB_CLS_SAP_SALES_HEAD' starts on the version '0.1' with the context 'Default'.

[INFO ]:xxxxxxxxxxxxxxxx.job_db_sap_sales_order_scheduling_0_1.JOB_DB_SAP_SALES_ORDER_SCHEDULING - tRunJob_3 - The child job 'JOB_CLS_SAP_SALES_HEAD' is done.

[INFO ]:xxxxxxxxxxxxxxxx.job_cls_sap_sales_head_0_1.JOB_CLS_SAP_SALES_HEAD - TalendJob: 'JOB_CLS_SAP_SALES_HEAD' - Start.

[INFO ]:xxxxxxxxxxxxxxxx.job_cls_sap_sales_head_0_1.JOB_CLS_SAP_SALES_HEAD - Got 2 Spark libjars

[INFO ]:xxxxxxxxxxxxxxxx.job_cls_sap_sales_head_0_1.JOB_CLS_SAP_SALES_HEAD - Spark libjars : /data/talend7.2/Talend-7.2.1/jobserver/agent/TalendJobServersFiles/repository/DIGITAL_FACTORY_PHASE_I_JOB_DB_SAP_SALES_ORDER_SCHEDULING_20210526_161905_eCDws/JOB_DB_SAP_SALES_ORDER_SCHEDULING/../../../cache/lib/2427524776_551552229/routines.jar,../../../cache/lib/3704464275_3101559666/job_cls_sap_sales_head_0_1.jar

Caused by: java.lang.ClassNotFoundException: org.talend.bigdata.dataflow.hmap.PostProcessor

at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

0695b00000F97h5AAB.png

Labels (4)
1 Reply
Anonymous
Not applicable

Hello,

We have replied to your another topic: https://community.talend.com/s/question/0D55b0000681fgmCAA/unix-shell-without-classpathreference-of-...

Could you please stick to one topic so that we can pay individual attention to your issue. Thanks for your understanding.

Best regards

Sabrina