Getting java.lang.NoClassDefFoundError for custom component on jobserver

[Anonymous]

We successfully created a custom component to read from rabbitMQ using the new framework

The component works in the studio 7.0.1.

But when we run the job on the jobserver we get an error - see log below;

Are there any additional steps when we publish/deploy custom components?

### Job STARTED at 2018/12/09 15:28:08 (jobId=20181209_152803_5VNKG, jobExecutionId=20181209152808_pGOkV) ###
[statistics] disconnected
[statistics] disconnected
Exception in thread "main" java.lang.NoClassDefFoundError: com/rabbitmq/client/DefaultConsumer
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetPublicMethods(Class.java:2902)
at java.lang.Class.getMethods(Class.java:1615)
at org.talend.sdk.component.runtime.manager.asm.ProxyGenerator.hasInterceptors(ProxyGenerator.java:491)
at org.talend.sdk.component.runtime.manager.ComponentManager.handleProxy(ComponentManager.java:921)
at org.talend.sdk.component.runtime.manager.ComponentManager.access$1400(ComponentManager.java:159)
at org.talend.sdk.component.runtime.manager.ComponentManager$Updater.lambda$onCreate$14(ComponentManager.java:1076)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
at java.util.LinkedList$LLSpliterator.forEachRemaining(LinkedList.java:1235)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at org.talend.sdk.component.runtime.manager.ComponentManager$Updater.onCreate(ComponentManager.java:1068)
at org.talend.sdk.component.container.ContainerManager$ContainerBuilder.lambda$null$1(ContainerManager.java:370)
at org.talend.sdk.component.container.ContainerManager.safeInvoke(ContainerManager.java:144)
at org.talend.sdk.component.container.ContainerManager.access$1100(ContainerManager.java:54)
at org.talend.sdk.component.container.ContainerManager$ContainerBuilder.lambda$create$2(ContainerManager.java:370)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174)
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.talend.sdk.component.container.ContainerManager$ContainerBuilder.create(ContainerManager.java:371)
at org.talend.sdk.component.runtime.manager.ComponentManager.addPlugin(ComponentManager.java:711)
at org.talend.sdk.component.runtime.manager.ComponentManager$3.lambda$new$3(ComponentManager.java:377)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at org.talend.sdk.component.runtime.manager.ComponentManager$3.<init>(ComponentManager.java:377)
at org.talend.sdk.component.runtime.manager.ComponentManager.instance(ComponentManager.java:349)
### Job ENDED WITH ERROR at 2018/12/09 15:28:09 (jobId=20181209_152803_5VNKG, jobExecutionId=20181209152808_pGOkV) ###

at talend.prollyaanvraag_0_1.pRollyAanvraag.AMQPInput_1Process(pRollyAanvraag.java:1148)
at talend.prollyaanvraag_0_1.pRollyAanvraag.tLibraryLoad_1Process(pRollyAanvraag.java:672)
at talend.prollyaanvraag_0_1.pRollyAanvraag.runJobInTOS(pRollyAanvraag.java:2240)
at talend.prollyaanvraag_0_1.pRollyAanvraag.main(pRollyAanvraag.java:1977)
Caused by: java.lang.ClassNotFoundException: com.rabbitmq.client.DefaultConsumer
at org.talend.sdk.component.classloader.ConfigurableClassLoader.loadClass(ConfigurableClassLoader.java:218)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 52 more

[Anonymous]

Hello,
It seems that you have a missing dependency in your project. the one that should provide the class `com/rabbitmq/client/DefaultConsumer`.
Can you check your dependencies and ensure you have all your required dependencies in compile scope.
if you plan to use any provided dependencies ensure they are provided by the framework or by the running platform (job server).

An easy way to check your job dependencies is to build your job from the studio by  right click on the job => Build Job and check the lib folder to ensure your jars files are all in there.

[Anonymous]

Anas,

Thanks for getting back.

I can run the job on the client machine and then it works.

But when I deploy the job from the TAC to the jobserver, it fails.

This is the .bat file on the jobserver

%~d0
cd %~dp0
D:\Java\jdk1.8.0_191\bin\java.exe -Xms256M -Xmx1024M -cp .;../../../cache/lib/3569584080_551552229/routines.jar;../../../cache/lib/2515122170_1359808236/amqp-client-5.4.3.jar;../../../cache/lib/2053048282_4182183202/advancedPersistentLookupLib-1.2.jar;../../../cache/lib/3434323647_3256683220/amqp-component-0.0.3-SNAPSHOT.jar;../../../cache/lib/1336530737_2793933396/commons-collections-3.2.2.jar;../../../cache/lib/242457337_1697318851/component-api-0.0.4.jar;../../../cache/lib/1086752861_2225934399/component-runtime-design-extension-0.0.5.jar;../../../cache/lib/746244976_2978155144/component-runtime-di-0.0.5.jar;../../../cache/lib/273176393_3420392301/component-runtime-impl-0.0.5.jar;../../../cache/lib/3859498358_4097051798/component-runtime-manager-0.0.5.jar;../../../cache/lib/2549687820_1712981974/component-spi-0.0.5.jar;../../../cache/lib/2754408678_1847134243/container-core-0.0.5.jar;../../../cache/lib/2178828168_605815915/dom4j-1.6.1.jar;../../../cache/lib/1972797759_4275703019/geronimo-annotation_1.3_spec-1.0.jar;../../../cache/lib/1732982708_2886797928/geronimo-json_1.1_spec-1.0.jar;../../../cache/lib/1039009832_3090090697/geronimo-jsonb_1.0_spec-1.0.jar;../../../cache/lib/2144549254_647365790/jaxen-1.1.1.jar;../../../cache/lib/2580262047_1866991928/jboss-serialization.jar;../../../cache/lib/2295871601_1549535082/johnzon-core-1.1.7.jar;../../../cache/lib/941375462_1716848605/johnzon-jsonb-1.1.7.jar;../../../cache/lib/1833419795_1878591558/johnzon-mapper-1.1.7.jar;../../../cache/lib/2437169554_1109001852/jtds-1.3.1-patch.jar;../../../cache/lib/2065431124_674301086/log4j-1.2.15.jar;../../../cache/lib/1015062404_674628767/log4j-1.2.16.jar;../../../cache/lib/3107690740_1118897678/slf4j-api-1.7.25.jar;../../../cache/lib/2601911240_1579091735/slf4j-log4j12-1.7.25.jar;../../../cache/lib/221808531_3912436596/talend_DB_mssqlUtil-1.2-20171017.jar;../../../cache/lib/237199807_3543468909/talend_file_enhanced_20070724.jar;../../../cache/lib/1009653832_312279964/trove.jar;../../../cache/lib/849693260_1937901690/xbean-asm6-shaded-4.7.jar;../../../cache/lib/1858677389_2343373179/xbean-finder-shaded-4.7.jar;../../../cache/lib/1125810284_1424230226/xbean-reflect-4.7.jar;../../../cache/lib/1420748432_1722746990/prollyaanvraag_0_1.jar;../../../cache/lib/1908744071_2077034843/prollyinformatie_0_1.jar talend.prollyaanvraag_0_1.pRollyAanvraag --context=PY910 %* --resuming_logs_dir_path=D:\Talend\TalendJobServersFiles\jobexecutions/logs\20181214155850_YKTnb\resuming_20181214155850_YKTnb.log --pid=20181214155850_YKTnb --client_host=10.30.2.117 --log4jLevel=Info

It has all the jars but somehow it does not recognise them.

Can it be related to the new component framework, because we used it to build a custom component?

[Anonymous]

Hello,
Thank you for sharing your class path.

I can see from your class path that you are using an old version (0.0.4) of the framework. This version is having a known issue about loading dependencies from the job lib folder that was fixed in 1.0.4
You can find the framework changelog in the doc : https://talend.github.io/component-runtime the issue is referenced in Talend jira under the id TCOMP-922.

I strongly suggest you to upgrade the framework version to get the fix.

Direct link to the changelog for the latest released version https://talend.github.io/component-runtime/main/1.1.4/changelog.html

You can also check the compatibility matrix (studio/framework) https://talend.github.io/component-runtime/main/1.1.4/compatibility.html

[youssef2]

HI,

I have the same Issue:

Everything is ok with my new component on the studio. 

It's also working when I launch the job manuelly from the exported nexuse release, using the .bat script.

But when I deploy the nexus release on Tac/jobserver, I have the execption java.lang.NoClassDefFoundError: com/google/api/client/http/AbstractInputStreamContent.

I'm using the version 1.1.1 of the framework wich is compatible with my studio Talend MDM platform 7.1.1.

Any solution/workaround?

Thanks.

Youssef

[youssef2]

I've tried to use diffrent versions of the framework but Tac says that it can't deploy jobs comming from a non supported version of studio...

I endup using a tjava with the component logic as an external library.

It's very annoying to waste time understanding and developping over a framework that can't go on production. 

[Anonymous]

Hello,

Have you already checked the configuration on the TAC navigating to command line and make sure it is pointing to the folder the custom component is on? Please ensure that you copy the entire custom component folder to that directory.

Best regards

Sabrina

[Sanjay5]

Hi All,

I have faced similar issue with Talend 7.3.1 with tCouchbaseInput component.

I am using Talend Bigdata 7.3.1 and running a job with tCouchbaseInput --> tLogRow

1. When i run the job in Studio, It is working without any issue
2. When i run the same job in Job Server, It is failing with the error
3. When i remove couchbase component from my job, It works fine in Job Server.

Error: 

Exception in thread "main" java.lang.NoClassDefFoundError: com/couchbase/client/java/env/CouchbaseEnvironment 

Any suggestions on how to resolve the issue.