Qlik Replicate: Kafka Message delivery failed Broker Broker failed to validate record

[David_Apodaca]

This article covers a specific issue encountered with Qlik Replicate using a Kafka target endpoint, where CDC (Change Data Capture) messages fail due to a mismatch in the configured message size limit between the Kafka broker or topic and Qlik Replicate. 

In a Qlik Replicate task using Kafka as a target connection, the following error is observed: 

00005828: 2025-08-06T23:51:01 [TARGET_APPLY ]W: rdkafka error: (code=87) 'Broker: Broker failed to validate record'. Replicate status: 1020414 (kafka_client.c:218)
00005828: 2025-08-06T23:51:01 [TARGET_APPLY ]E: Kafka: Message delivery failed: Broker: Broker failed to validate record. [1020414] (kafka_client.c:273)

Resolution

1. Verify the Kafka broker and topic configurations:
   • Check max.message.bytes on the Kafka broker
   • Check message.max.bytes on the Kafka topic
2. In Qlik Replicate:
   1. Open the Kafka target endpoint settings
   2. Go to the 'Advanced' section
   3. Locate and update the 'Message maximum size' parameter to match the Kafka configuration (e.g., 2,097,164 bytes)
   4. Restart the task if necessary to apply the new configuration.
   5. Monitor the logs to confirm that large messages are now being sent successfully without errors.

Best Practice Recommendation

Always ensure the Message maximum size in Qlik Replicate is set to equal or slightly below the lowest value configured on the Kafka broker or topic. This ensures that Qlik Replicate will not attempt to send messages that Kafka cannot accept, avoiding potential task failures.

Cause

This Kafka error indicates the broker determined that the record did not meet certain requirements. One possible example is explained in this article.

Kafka enforces a maximum message size using two parameters:

• max.message.bytes on the broker level
• message.max.bytes on the topic level

In Qlik Replicate, the Kafka target endpoint also has an advanced setting named Message maximum size. If a message exceeds this configured size, Qlik Replicate will fail to send it, even if Kafka is technically capable of accepting it.

Example:

• Kafka was configured to accept messages up to 2,097,164 bytes
• Qlik Replicate's 'Message maximum size' was set to 2,097,152 bytes

Because Qlik Replicate’s threshold was slightly lower, messages at the upper boundary could be silently dropped or fail to process correctly.

Environment

• Qlik Replicate
• Kafka Endpoint