topic Re: tkafkaInput Advanced settings and error handling in Talend Studio

tkafkaInput Advanced settings and error handling

sensiva — Sat, 16 Nov 2024 08:24:08 GMT

Hello,

I am currently working with the tkafka components and for the moment all is working on a success scenario, but i am facing lot of issues on the error handling. For example the advanced settings, i gave ack=all, retries=10, but none of these seems to get considered. I tried to test a scenario where i brought down a broker with ack=all, but the kafka input doesn't try to turn down that transaction rather continued to process the next message and finished with success, i had message lost during this transaction.

Is this a bug ? or there are other ways of configuring these additional properties ?

Would be really helpful with some pointers on this topic.

Thanks

Re: tkafkaInput Advanced settings and error handling

Anonymous — Thu, 19 Apr 2018 08:59:06 GMT

Hello,

For further information about the consumer properties you can define in Kafka consumer properties, please see the section describing the consumer configuration in Kafka's documentation in http://kafka.apache.org/documentation.html#consumerconfigs.

Best regards

Sabrina

Re: tkafkaInput Advanced settings and error handling

Anonymous — Sat, 24 Nov 2018 08:51:12 GMT

Hi,

Am getting the following error "Exception in component tKafkaInput_1
kafka.common.MessageSizeTooLargeException: Found a message larger than the maximum fetch size of this consumer on topic "

I have increased my fetch size as attached . Please assist .

kafka.PNG

Re: tkafkaInput Advanced settings and error handling

sensiva — Thu, 20 Dec 2018 15:17:25 GMT

I see the properties being defined in the kafka node, are they inline with
the properties defined in the kafka server ? May be that could be a point
to look

Re: tkafkaInput Advanced settings and error handling

Anonymous — Fri, 21 Dec 2018 09:04:15 GMT

Hello @jibe

We see that this topic is set as resolved.

https://community.talend.com/t5/Design-and-Development/tkafkaInput-Advanced-settings/m-p/142543

Is this solution "clear your additional Kafka properties as in your attached config and try using the "fetch.message.max.bytes" parameter" Ok with you?

Best regards

Sabrina

Re: tkafkaInput Advanced settings and error handling

Anonymous — Fri, 21 Dec 2018 09:13:36 GMT

Hello @sensiva

What's kafka API version are you using? The kafka API changed between 0.8 and 0.9.

Best regards

Sabrina