If we increase the logstream retention from 72 hours to 144 hours, the main impact will be on disk usage. Since we are doubling the retention window, storage consumption will also increase accordingly based on the volume of logs being generated. CPU and memory impact should be minimal, as these are more dependent on active processing rather than how long the logs are retained.
From a performance standpoint, having a longer retention means more data is available for recovery, but it could also slightly increase the time needed if a task has to catch up from an older point. Cleanup operations might also take a bit longer depending on the volume.
Before making this change, it would be good to ensure there is enough disk space available and to keep an eye on storage growth after the update. As long as storage is planned properly, increasing the retention should not cause any major issues.
I have a question regarding how Log Stream retention works in Qlik Replicate.
If I configure:
Retention time = 48 hours
Size limit = 100,000 MB
What happens in the following scenario?
After 48 hours, the total log stream size is only 50,000 MB (i.e., below the configured size limit)
Will Replicate:
Delete the files immediately because the time limit is reached, or
Retain them until the size limit is also reached?
In other words, do the time and size settings work as an “OR” condition, or is there any priority between them?
The reason I’m asking is to understand whether we can retain additional changes in the Log Stream staging layer when the size limit has not yet been reached.
Additionally, I would like to clarify:
If a log stream file is older than 48 hours, but a child task is still reading it, will Replicate still delete the file, or will it wait until the read operation is completed?
Any clarification on this behavior would be very helpful.
As per the Qlik Replicate behavior, the retention settings work with two independent mechanisms. The time-based setting controls how long staging files are retained, and files older than the configured duration are eligible for deletion. The size-based setting acts as a threshold mechanism and is triggered only when the total size of staging files exceeds the defined limit, at which point the oldest files are deleted until the size falls below the threshold.
Retention: Specify when the staging file should be deleted (note that active files will not be deleted):
Delete staging files after (hours):Select the check box and specify the maximum time before a file is deleted. Default is 48 hours. Maximum permitted time is 10,000 hours.
Delete oldest files when the total size of all staging files exceeds (MB):The maximum size that you want to allocate for the staging folder. If the specified size is reached,Replicatewill start deleting files from the oldest to the newest until the total size falls below the upper limit.
In your scenario, even if the total size is below the configured limit, files older than the retention period (48 hours) will still be deleted.
However, as noted in the documentation, active files are not deleted. This means that if a file is still being read by a task (e.g., a child task), Replicate will retain it even if it exceeds the retention time, and deletion will occur only after it is no longer in use.
In summary, time-based retention acts as the primary control, size-based cleanup is conditional, and active usage always takes precedence over deletion.
