General understanding of Qlik Replicate Change Processing Tuning and Batch Tuning

[Steve_Nguyen]

The Longer than (seconds) parameter is used to determine how long to wait for the change capture to become quiescent so that we can upload what we have captured. We can refer to this as the CDC Idle Timeout.
 
The But less than (seconds) parameter is used to determine the maximum amount of time to wait before sending what is ready, regardless of if the CDC Idle Timeout has been reached.
 
The Force apply a batch when processing memory exceeds (MB) parameter is used to determine the maximum amount of data to build up in a batch before sending, regardless of the two time based parameters.
 
Usage scenarios:

• When changes are actively arriving from the source then all of the parameters are used.
• When changes are not actively arriving from the source and there is a backlog of changes waiting to go to the target then batches are built and sent when the ‘But less than’ time has been reached OR the ‘force apply a batch…’ size has been reached.

 
The settings come down to what the latency requirements are and the speed in which the target can accept batches. If the target can handle small batches efficiently then the default values are usually used. If the target prefers large batches (Teradata, Vertica, Greenplum, Snowflake) then increasing the size of the ‘Longer than…’ and ‘Force apply…’ parameters can be tuned based on the speed of the transfer to the target.

Transactions can be cached to disk in swap files (.tswp) in the ..\data\tasks\TASK_NAME\sorter folder if there are large or long running transactions on the source, or if the target cannot keep up with the data being sent. The Transaction Offload Tuning settings determine when changes are cached to disk, when reaching the size or duration values set, whichever comes first.

Environment:

Qlik Replicate 

[kishetty03]

so if I want my task to apply the changes captured in last 30 mins only. just like scheduling every 30 mins unless processing memory reach  then waht should be  longer than and less than seconds @Steve_Nguyen 

[Sonja_Bauernfeind]

Hello @kishetty03 

Please post your query in the Qlik Replicate forum, indicating your requirements and what you are attempting to achieve. You will have access to our active community and support engineers in this forum.

All the best,
Sonja