Unable to publish or fetch data from Kafka

[Anonymous]

Hi

I have build job in Talend integration cloud hybrid edition, create kafka topic and publish data. Below mapping.

I am able to create topic in kafka but when i try to consume data in the terminal, it doesn't show any data.

Similar issue with kafka consumer tkafkainput as well.

tkafkainput : always keep running inspite of setting stop after 6000ms.

 

Kafka versions in sync.

[Anonymous]

Hello,

The topic you created in kafka is Ok with you? Could you please indicate on which build version you got this issue?

Here is a jira issue about tKafkaOutput component does not implement "use existing connection"

:https://jira.talendforge.org/browse/TBD-3435

Can you use tkafkaInput(Built-in)-->tlogrow to read your data without using tKafkaConnection component?

Best regards

Sabrina

[Anonymous]

Hello,

I tried without tkafkaconnection but No luck job keeps running without fetching any data.

Talend version: Tested in 2 products

Talend Integration Cloud Hybrid Edition

Version: 6.4.1
Build id: 20170623_1246

----

Talend Open Studio for Big Data

Version: 6.3.1

Regards,

M

[Anonymous]

Hello,

What is the job you are building? Show me how you build specificially

[Anonymous]

Hello,

Would you mind posting your tkafkainput component setting screenshot into forum?

The tKafkaInput is a streaming component, even in the DI palette.

Could you please try use tfileoutpudelimited component as output and check the option "use output stream" on the tFileOutputDelimited to correctly put the data into the file?

Best regards

Sabrina

[Anonymous]

Hello,

I am using tkafkaInput and connecting it with tLogRow, when I run this job, messages are read but not printed in tLogRow. Can you please point me the missing setting here.

 I'm using kafka version 0.9.0.1 and Talend Open Studio for Big Data (6.4.1) Free Edition

Thanks in advance.

[Anonymous]

Hi

The tKafkaInput is a streaming component.

If you want to save data to a file with tkafkaInput, you have to check the option "use output stream" on the tFileOutputDelimited to correctly put the data into the file.

Let us know if this solution is OK with you.

Best regards

Sabrina

[Anonymous]

I did as you suggested but i am getting below error .

Exception in component tFileOutputDelimited_1 (kafka)
java.lang.NullPointerException
at java.io.Writer.<init>(Unknown Source)
at java.io.OutputStreamWriter.<init>(Unknown Source)
at local_project.kafka_0_1.kafka.tKafkaInput_1Process(kafka.java:947)
at local_project.kafka_0_1.kafka.tKafkaConnection_1Process(kafka.java:586)
at local_project.kafka_0_1.kafka.runJobInTOS(kafka.java:1613)
at local_project.kafka_0_1.kafka.main(kafka.java:1464)
[statistics] disconnected

Job kafka ended at 16:19 31/01/2019. [exit code=1]

I attached screnn shots of my component settings.

Please help me on this.

Thanks,

Ganga

screen_shots.docx