Topics retention time

[Gladi]

Hi

I have a problem with settings in Qlik Replication.

We replicate db2 tables to confluent kafka. The retention time of the topics is one week now but I'd like to increase the retention time to three weeks.

But how to implement those changes in Qlik replication task?

Can you help me? 

Thank you in advance 

Gladi

[SwathiPulagam]

Hi @Gladi ,

We cannot change the topic retention policy from the replicate side. It can be done by your Kafka admin team on the topic level.

Thanks,

Swathi

[Gladi]

Hi Swathi,

There aren't target internal parametersi for topic retention policy in Qlik?

Thanks

Gladi

[SwathiPulagam · Accepted solution]

Hi @Gladi ,

        Replicate use librdkafka client API to connect Kafka and it exposed only limited parameters to use(delete.retention.ms is not in the list of exposed parameters so we can't use it from replicate).

https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md

if you want to use rdkafka Topic configuration properties then you can pass those parameters using rdkafkaTopicProperties internal property. In the same way, if you want to use rdkafka Global configuration properties then you can pass those parameters using rdkafkaProperties internal parameter.

Thanks,

Swathi

[Heinvandenheuvel]

There are no 'documented' internal parameters to specify topic retention which is clearly a Kafka thing, far away from Replicate's main function.

You could try to find something useful to stick into: "additionalConnectionProperties" or "rdkafkaProperties"

Hein.