Search or browse our knowledge base to find answers to your questions ranging from account questions to troubleshooting error messages. The content is curated and updated by our global Support team
The Longer than (seconds) parameter is used to determine how long to wait for the change capture to become quiescent so that we can upload what we have captured. We can refer to this as the CDC Idle Timeout.
The But less than (seconds) parameter is used to determine the maximum amount of time to wait before sending what is ready, regardless of if the CDC Idle Timeout has been reached.
The Force apply a batch when processing memory exceeds (MB) parameter is used to determine the maximum amount of data to build up in a batch before sending, regardless of the two time based parameters.
Usage scenarios:
- When changes are actively arriving from the source then all of the parameters are used.
- When changes are not actively arriving from the source and there is a backlog of changes waiting to go to the target then batches are built and sent when the ‘But less than’ time has been reached OR the ‘force apply a batch…’ size has been reached.
The settings come down to what the latency requirements are and the speed in which the target can accept batches. If the target can handle small batches efficiently then the default values are usually used. If the target prefers large batches (Teradata, Vertica, Greenplum, Snowflake) then increasing the size of the ‘Longer than…’ and ‘Force apply…’ parameters can be tuned based on the speed of the transfer to the target.
Transactions can be cached to disk in swap files (.tswp) in the ..\data\tasks\TASK_NAME\sorter folder if there are large or long running transactions on the source, or if the target cannot keep up with the data being sent. The Transaction Offload Tuning settings determine when changes are cached to disk, when reaching the size or duration values set, whichever comes first.
Environment:
