Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Want to see what's currently in public preview? We've pulled it all together in one place. Check it out here
cancel
Showing results for 
Search instead for 
Did you mean: 
Eddie_Liu
Contributor
Contributor

Spark libjars can't be gotten

we have following job, this job will submit to spark yarn. JOB_CLS_SAP_SALES_HEAD is a big data batch. when we deploy the job, we found only 2 Spark libjars were got, actually it need more libjars, it made NoClassDefFoundError. what's wrong? could anyone help on it?

[INFO ]:xxxxxxxxxxxxxxxx.job_db_sap_sales_order_scheduling_0_1.JOB_DB_SAP_SALES_ORDER_SCHEDULING - tRunJob_2 - The child job 'digital_factory_phase_i.job_stg_sap_vbap_0_1.JOB_STG_SAP_VBAP' is done.

[INFO ]:xxxxxxxxxxxxxxxx.job_db_sap_sales_order_scheduling_0_1.JOB_DB_SAP_SALES_ORDER_SCHEDULING - tRunJob_3 - The child job 'JOB_CLS_SAP_SALES_HEAD' starts on the version '0.1' with the context 'Default'.

[INFO ]:xxxxxxxxxxxxxxxx.job_db_sap_sales_order_scheduling_0_1.JOB_DB_SAP_SALES_ORDER_SCHEDULING - tRunJob_3 - The child job 'JOB_CLS_SAP_SALES_HEAD' is done.

[INFO ]:xxxxxxxxxxxxxxxx.job_cls_sap_sales_head_0_1.JOB_CLS_SAP_SALES_HEAD - TalendJob: 'JOB_CLS_SAP_SALES_HEAD' - Start.

[INFO ]:xxxxxxxxxxxxxxxx.job_cls_sap_sales_head_0_1.JOB_CLS_SAP_SALES_HEAD - Got 2 Spark libjars

[INFO ]:xxxxxxxxxxxxxxxx.job_cls_sap_sales_head_0_1.JOB_CLS_SAP_SALES_HEAD - Spark libjars : /data/talend7.2/Talend-7.2.1/jobserver/agent/TalendJobServersFiles/repository/DIGITAL_FACTORY_PHASE_I_JOB_DB_SAP_SALES_ORDER_SCHEDULING_20210526_161905_eCDws/JOB_DB_SAP_SALES_ORDER_SCHEDULING/../../../cache/lib/2427524776_551552229/routines.jar,../../../cache/lib/3704464275_3101559666/job_cls_sap_sales_head_0_1.jar

Caused by: java.lang.ClassNotFoundException: org.talend.bigdata.dataflow.hmap.PostProcessor

at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

0695b00000F97h5AAB.png

Labels (4)
1 Reply
Anonymous
Not applicable

Hello,

We have replied to your another topic: https://community.talend.com/s/question/0D55b0000681fgmCAA/unix-shell-without-classpathreference-of-...

Could you please stick to one topic so that we can pay individual attention to your issue. Thanks for your understanding.

Best regards

Sabrina