topic Re: Exactly-Once in Replicate Kafka Target in Qlik Replicate

Exactly-Once in Replicate Kafka Target

bobhaffner — Fri, 07 Feb 2020 15:40:11 GMT

Does Attunity Replicate support Exactly-Once when publishing to a Kafka topic?

ie "enable.idempotence","true"

Re: Exactly-Once in Replicate Kafka Target

Madhavi_Konda — Tue, 11 Feb 2020 11:37:32 GMT

Hi,

Replicate uses librdkafka APIs and these properties can be configured in the replicate console.
By default "enable.idempotence","false" but, as I said it is configurable and you can make it as "true"

The parameters listed in the below librdkafka URL are used by Replicate.
https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md

Thanks,
Madhavi

Re: Exactly-Once in Replicate Kafka Target

bobhaffner — Wed, 12 Feb 2020 14:31:17 GMT

Hi Madhavi,

I tried to set that property in rdkafkaProperties alongside my other properties, but I got the following error

SYS-E-HTTPFAIL, Failed to set rdkafka property 'enable.idempotence': No such configuration property: "enable.idempotence"..

What version of librdkafka is Attunity 6.5.0.354 using?

Thanks

Bob

Re: Exactly-Once in Replicate Kafka Target

Madhavi_Konda — Tue, 18 Feb 2020 13:58:28 GMT

Hi Bob,
Good day!
You need to use rdkafkaTopicProperties internal parameter to enable this setting and replicate supporting librdkafka from the beginning of the Kafka target endpoint introduction in the replicate.

Thanks,
Madhavi

Re: Exactly-Once in Replicate Kafka Target

bobhaffner — Wed, 22 Apr 2020 12:18:26 GMT

Turns out that Replicate does not support the property enable.idempotence after all