topic Topics retention time in Qlik Replicate

Topics retention time

Gladi — Fri, 25 Feb 2022 16:40:24 GMT

I have a problem with settings in Qlik Replication.

We replicate db2 tables to confluent kafka. The retention time of the topics is one week now but I'd like to increase the retention time to three weeks.

But how to implement those changes in Qlik replication task?

Can you help me?

Thank you in advance

Gladi

Re: Topics retention time

SwathiPulagam — Fri, 25 Feb 2022 16:50:39 GMT

Hi @Gladi ,

We cannot change the topic retention policy from the replicate side. It can be done by your Kafka admin team on the topic level.

Thanks,

Swathi

Re: Topics retention time

Gladi — Fri, 25 Feb 2022 17:13:00 GMT

Hi Swathi,

There aren't target internal parametersi for topic retention policy in Qlik?

Thanks

Gladi

Re: Topics retention time

SwathiPulagam — Fri, 25 Feb 2022 17:45:19 GMT

Hi @Gladi ,

Replicate use librdkafka client API to connect Kafka and it exposed only limited parameters to use(delete.retention.ms is not in the list of exposed parameters so we can't use it from replicate).

https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md

if you want to use rdkafka Topic configuration properties then you can pass those parameters using rdkafkaTopicProperties internal property. In the same way, if you want to use rdkafka Global configuration properties then you can pass those parameters using rdkafkaProperties internal parameter.

Thanks,

Swathi

Re: Topics retention time

Heinvandenheuvel — Fri, 25 Feb 2022 18:25:58 GMT

There are no 'documented' internal parameters to specify topic retention which is clearly a Kafka thing, far away from Replicate's main function.

You could try to find something useful to stick into: "additionalConnectionProperties" or "rdkafkaProperties"

Hein.