Changes accumulating on disk but are not being applied to the target

[MoeyE]

Hi guys,

One of our clients was experiencing high latency for the task and we found out that his stream buffer was full so we increased the size to 

"stream_buffers_number" : 10,
"stream_buffer_size" : 20,

This succeeded in reducing the latency however that task was not applying any changes to the target. Pictured below, the "On Disk" bar is increasing higher and higher while the "In Memory" bar is very low almost 0. I am checking the swap files for the task and the size of one of the files is increasing, in total there's maybe 7 files. The biggest one was about 180,000kb and rising. From my understanding, this means that there is 1 extremely large transaction stored on the disk but for some reason it is not getting applied to the target. Could there be something to do with the change processing tuning settings that is causing this to occur? I can supply the task export to a support member directly if required, for security purposes I won't leave it on this ticket.

We already turned up TARGET_APPLY, SOURCE_CAPTURE, and STREAM but so far we have found nothing to indicate the issue. We also looked at SORTER and I noticed a No more events to forward message. In the meantime, I have the task running and will wait approximately 30 minutes before I go back in to change the stream settings back to default and come up with a new plan. Also, the task takes approx 30 minutes to stop. I see that this is because of unfinished transactions.

(This is not a screenshot of the actual task's console, All the non-highlighted bars are meant to be empty)

Any help or ideas on this is appreciated. 

Kind regards,

Mohammed

[boaz_newman]

Hello

Please open a support case, this issue needs analysis of log files, etc.

Regards

boaz

[MoeyE]

Thanks