topic Talend ESB 6.2.1 - kafka cMessageEndpoint errors in Talend Studio

Talend ESB 6.2.1 - kafka cMessageEndpoint errors

vapukov — Sat, 16 Nov 2024 10:20:55 GMT

Hello,
I have a few Jobs in Talent Open Studio for BigData connecting with kafka - everything fine and stable
as well - few routes with ActiveMQ and RabbitMQ in Talent ESB
But when I try to create new route for use same principles (and same connection details) with kafka - I have error:
"Exception in thread "main" java.lang.NoClassDefFoundError: scala/Product
at java.lang.ClassLoader.defineClass1(Native Method) ... "

URI - "kafka:192.168.20.107:9092?topic=t1&zookeeperHost=192.168.20.107&zookeeperPort=2181&groupId=uwr1"
component cKafka work, but unstable - can read portions of messages, than freeze (may be it settings problem, now I looking for other error)
Would be thankful for any ideas for check
Best regards, Vlad

Re: Talend ESB 6.2.1 - kafka cMessageEndpoint errors

Anonymous — Tue, 11 Oct 2016 09:40:56 GMT

Hi,
What does your new route look like? Could you please show us your component setting screenshot which will be helpful for us to address your issue?
Best regards
Sabrina

Re: Talend ESB 6.2.1 - kafka cMessageEndpoint errors

vapukov — Thu, 13 Oct 2016 13:08:52 GMT

Hi,
What does your new route look like? Could you please show us your component setting screenshot which will be helpful for us to address your issue?
Best regards
Sabrina

Hi Sabrina,
All in original message, but screenshots attached
and consumer and producer work fine with Kafka from Open Studio Big Data, but not from ESB

Re: Talend ESB 6.2.1 - kafka cMessageEndpoint errors

vapukov — Thu, 13 Oct 2016 13:12:24 GMT

if use dedicated Kafka component in ESB - it work, but:
1) only once - if route started after producer
2) not always receive all messages
3) no visible errors, but job or work or not, and never not work for new messages coming after start

Re: Talend ESB 6.2.1 - kafka cMessageEndpoint errors

Anonymous — Sat, 04 Mar 2017 18:53:35 GMT

Hi I can's see cMessageEndpoint in ESP Open Studio, neither in Data Integrator.
Appreciate if you guide me on how to get these additional components.

-Regards

Re: Talend ESB 6.2.1 - kafka cMessageEndpoint errors

vapukov — Sun, 05 Mar 2017 07:51:19 GMT

Hi I can's see cMessageEndpoint in ESP Open Studio, neither in Data Integrator.
Appreciate if you guide me on how to get these additional components.

-Regards

it's name - cMessagingEndpoint and it part of Talend Open Studio for ESB when You create new route

Re: Talend ESB 6.2.1 - kafka cMessageEndpoint errors

vapukov — Sun, 05 Mar 2017 07:55:41 GMT

by the way - I not resolve this problem, and as solution was chosen ActiveMQ as much stable
difference in performance on our tasks not so critical
will have plan re-test it in feature, with 1.0.

Re: Talend ESB 6.2.1 - kafka cMessageEndpoint errors

Anonymous — Sun, 05 Mar 2017 18:40:58 GMT

@palas - you need to switch to Mediation view in Talend Studio
@vapukov - It would be really helpful if you raise a bug about this and attach complete testing project as well.
On the other hand, from the message you have it looks quite clear that you are missing scala lib in your path. So either you didn't downloaded all dependencies for the component, or you have some buggy studio version (you can recompile from source yourself).
It is something related to:

 <dependency>
        <groupId>org.scala-lang</groupId>
        <artifactId>scala-library</artifactId>
        <version>xxx</version>
    </dependency>
    <dependency>
        <groupId>org.scala-lang</groupId>
        <artifactId>scala-xml</artifactId>
        <version>xxx</version>
    </dependency>

Could be caused by version bump as well.

Re: Talend ESB 6.2.1 - kafka cMessageEndpoint errors

Anonymous — Mon, 06 Mar 2017 18:14:43 GMT

Thanks @vapukov
I found it in Routes .
Now I am facing some issue:
My connection string is
"rabbitmq://'104.199.xxx.xxx:5672/amq.fanout?username=user&password=password&vhost=/&queue=test_queue&connectionTimeout=300"
I tied variation without vhost and connectionTimeout.
But, always same error:
Starting job RabbitMQ_Consumer1 at 23:42 06/03/2017.

connecting to socket on port 3649
connected
Failed to create route RabbitMQ_Consumer1_cMessagingEndpoint_1: Route(RabbitMQ_Consumer1_cMessagingEndpoint_1)

Re: Talend ESB 6.2.1 - kafka cMessageEndpoint errors

vapukov — Mon, 06 Mar 2017 21:05:39 GMT

generally - this topic about Kafka 🙂 and anybody will try to open is Kafka related
if You have problem with RabbitMQ - open (or add to old) related to RabbitMQ
by the way in Your case - all in error text:

"rabbitmq://'104.199.xxx.xxx:5672/amq.fanout?username=user&password=password&vhost=/&queue=test_queue&connectionTimeout=300"

Failed to resolve endpoint: rabbitmq://'104
You have wrong character in address - '