topic CDC Taking Time in Qlik Replicate

CDC Taking Time

Pranita123 — Thu, 12 Sep 2024 15:37:12 GMT

Hello Community,

We are facing performance issues while bulk insert and updates comes from source, as they too long to reflect on the target. we've checked both source and target availability, and everything seems fine, but the changes are accumulating at the target apply disk, as shown in the screenshot below.

we also enabled verbose logging for both source and target apply performance but haven't been able to identify any specific issue. any suggestions from our community member on how to resolve this would be greatly appreciated.

Thank you,

Pranita

Re: CDC Taking Time

DesmondWOO — Thu, 12 Sep 2024 16:04:25 GMT

Hi @Pranita123 ,

Based on your picture, the changes are accumulated on the target side and offloaded to the disk. You can enable verbose logging on the TARGET_APPLY to see how long Replicate takes to update the target. Please also check if you have a large transaction or if transactions are taking a long time to commit. If a transaction is too large or takes too long, Replicate will offload the changes to the disk. Once changes are offloaded, performance may be impacted.

Regards,
Desmond

Re: CDC Taking Time

john_wang — Fri, 13 Sep 2024 01:00:19 GMT

Besides @DesmondWOO comments:

1. make sure the Change Processing Mode is in "Batch Optimized apply"

2. if the monitored tables have PK, if no it impacts the applying performance too.

In a nutshell, the picture shows the target side cannot swallow the changes in a time fashion, the reasons maybe network bandwidth, target DB under heavy load, the table(s) has no PK etc.

Hope this helps.

John.