Official Support Articles

[lyka]

Sorter files are located on the data directory of the Replicate server.

Sample location: 

C:\Program Files\Attunity\Replicate\data\tasks\<task name>\sorter

The twsp files are transactions that have not been applied to target yet. They are automatically purged after use. 

Sorter files will accumulate if there if either:

• Big transactions from the source are not committed yet
• Target transactions are not applied yet
• Target transactions have been applied but no confirmation was returned

You cannot control the size of the sorter file however you can configure Replicate to use more memory and a longer transaction timeout.

Increasing the Transaction Offload Tuning values can help, especially the Transactions duration exceeds the default values. 

Locate the settings in:

1. Designer --> Task Settings --> Change Processing --> Change Processing Tuning -->
2. Transaction Offload Tuning Total transactions memory size exceeds (MB): 
   Set this to a larger memory allocation, such as 4G, 8G etc, depending on the total memory of the AR server.
3. Transactions duration exceeds (seconds): 
   Set to a longer time, eg 1 hour etc, depending on the transaction requirements. 

It is not advisable to delete the tswp files in the sorter file as this will impact Replicate's ability to process the changes. It acts as a landing zone for changes which have not been applied to the target, so deleting those files can cause unexpected behaviour and possibly data loss.

[JrQ]

Does Sorter files cleaned up after they are not needed?

[Sonja_Bauernfeind]

Hello @JrQ 

We can confirm that the .TWSP files are automatically purged by Replicate once consumed or have otherwise become invalid.

[cdc]

Hi Sonja,

Thank you before for the explanation, if our twsp file consist of not used task and not yet purged by Replicate, can we just delete them and will this action affect the Qlik Replicate?

Thank you,

[Sonja_Bauernfeind]

Hello @cdc 

Let me get clarification for you on this.

All the best,
Sonja 

[KellyHobson]

Hey @cdc ,

It is not advisable to delete tswp files in the sorter file as this will impact Replicate's ability to process the changes. It acts as a landing zone for changes which have not been applied to the target, so deleting those files can cause unexpected behavior and possibly data loss.

If your .twsp files are sitting in the sorter path of a non-use or non-active task, it is possible to delete them to clean up space. You need to ensure the task is completely stopped first. When you want to restart the task in the future you will either need to 1. Perform a reload of the task or 2. Restart from timestamp / or SCN;LSN depending on your source type, task settings, what data is already in your target.

So short answer: it is best practice to leave sorter path alone.

If you are in dire need of cleaning up space, you can do the manual deletes, but you will need to be prepared for a reload of the task. Additional note: If you stop the task and perform a reload or a restart from timestamp it will clean up the .twsp files and start "fresh" since it is doing a new query of the metadata.

Please let me know if you have any additional questions.

Best,

Kelly

[cdc]

Hi Kelly,

Thank you before,
Since it is not advisable to delete the tswp files, so if we are in a position of running out of memory, is there any recommendation to clean the space up?

Thank you,

[cdc]

Hi Kelly,

This question is a little out of the topic, but in case we are going to delete the tswp file, but the position of our task is disable which mean the task cannot be stopped.
Can we delete the tswp file in posisition of disable task?

Thank you,

[KellyHobson]

Hey @cdc 

For first question:

If you are running out of memory, you can stop the task and perform a restart from timestamp to clear out .twsp files.

See more information in this article:

https://community.qlik.com/t5/Knowledge/How-to-Start-Replicate-task-from-timestamp-or-bin-log-positi...

Additionally, the comments on this post are helpful about the sorter files:

https://community.qlik.com/t5/Qlik-Replicate/Qlik-replicate-task-folder-sorter/m-p/1918634#M2422

For second question:

It seems like there may be something going on if you are unable to properly stop the task. Do you plan to resume the task again if it is disabled? If you delete the files it will cause some missing data, so you will need to do a reload. I recommend opening a support case or posting on the community forum specific to why task getting disabled: https://community.qlik.com/t5/Qlik-Replicate/bd-p/qlik-replicate-discussions

Best,

Kelly