cKafka component not working in routes in Talend 8

[michelabarsottini]

Hi experts,

we are currently migrating routes and job from Talend ESB 6.2 Open Studio to Talend ESB 8 (R2024-07) Enterprise with Talend Administration Console.

The problem we are facing concerns the cKafka component in routes: when active the route is failing to deploy in TAC, but when we invoke the Kafka classes with custom java code everything is working fine (Kafka is up and collecting messages).

Reviewing the logs we think this is caused by the configuration parameters, here you can find the configurations in Talend 6.2 and the migrated component in Talend 8.

Talend 6.2:

Talend 8:

Does anyone knows how to setup the parameters and get this to work?

Best regards,

Michela 

[TalendDisaster]

Hi experts too,

I have exactly the same issue. Reading the documentation - https://help.talend.com/en-US/components/8.0/mediation-kafka/ckafka-standard-properties - they said : The following options are available only when the cKafka is used as a Producer. But yes I have the screenshot as you and there is no option how to set ckafka as a producer even if I update with null some value dedicated to consumer.

Regards,

Gregory

[michelabarsottini]

Hi Gregory,

we managed to find a solution!

We had to change the Partitioner/Serializer Class/Key Serializer Class parameters with the correct java classes and delete the kafka property "serializerClass", now it works perfectly!

Tomorrow I will attach the correct configuration that works on the TAC environment!

Best regards,

Michela

[TalendDisaster]

Hi Michela, great job ! that would be very helpful to share the correct configuration ! So I'm waiting tomorrow.

Regards,

Gregory

[michelabarsottini · Accepted solution]

Hi Gregory,

here's our solution, I hope it can be helpful:

in the Partitioner parameter "org.apache.kafka.clients.producer.internals.DefaultPartitioner"

in the Serializer Class and Key Serializer Class: "org.apache.kafka.common.serialization.StringSerializer" (same value for both).

We also had a conflict between the Kafka default library for the talend component and the library in the custom code in Global Routines, we had to unflag the "Required" field in the "Modify routine libraries" menu.

Hope it helps!

Kind regards,

Michela