topic How to use tKakfaInput component to consume the Topics generated via tKafkaOutput component in Talend Studio

How to use tKakfaInput component to consume the Topics generated via tKafkaOutput component

Anonymous — Thu, 13 Apr 2017 07:21:53 GMT

I have Talend job, where in I am using "tKafkaConnection", "tKafkaInput" and "tLogRow" components.
I am using Talend Open Studio for Big Data (6.3.1) Free Edition.

The value of Zookeeper Quorum list and Broker list for "tKafkaConnection" has been set properly for tKafkaConnection component.

Value of "Topic Name"has been set to "testKafkaTalend" and value of "Consumer Group Id" has been set to "Con_grp" for tKafkaInput component.
And then I have used "tLogRow" component.

I have built a Producer too in a Talend using tKafkaOutput component.

The problem that I am facing is the when I use the non-Talend Kafka Consumer (regular Kafka consumer running separately on the machine, and not use the tKafkaInput component), I can see the topics that are being produced via the Producer that I wrote.

But when I use the Talend based Kafka Consumer, I am not able to see any data being consumed.

Can anyone suggest what am I doing wrong with tKafkaInput?
Attached are the configurations of the various components.

Re: How to use tKakfaInput component to consume the Topics generated via tKafkaOutput component

Anonymous — Wed, 10 May 2017 04:08:11 GMT

Hello,

Is there any message printed on console when you use the Talend based Kafka Consumer(tKafkaInput component)?

Best regards

Sabrina

Re: How to use tKakfaInput component to consume the Topics generated via tKafkaOutput component

Anonymous — Wed, 10 May 2017 16:16:14 GMT

No, there is no message that is printed on console.

Re: How to use tKakfaInput component to consume the Topics generated via tKafkaOutput component

vapukov — Wed, 10 May 2017 23:39:27 GMT

version of Kafka same as You set in Talend?

Kafka is very sensitive for minor version changes

Re: How to use tKakfaInput component to consume the Topics generated via tKafkaOutput component

Anonymous — Thu, 11 May 2017 07:11:16 GMT

Yes, they are same in both producer and consumer.

For now, this has been resolved.

What I did was, created the whole job again, using the same components.

And it worked in the new job.

I am not able to see any difference in the old job and new job.
But it worked in the new job.