topic Re: Incomming - on disk; where does this information actually exist in Qlik Replicate

Incomming - on disk; where does this information actually exist

RonFusionHSLLC — Wed, 07 Jun 2023 14:46:24 GMT

I had a situation where the local disk where the product SQLLite db is stored filled.

Replication crashed, with millions of transactions in flight.

I remedied the issue and got replication caught up, however I still see this every day...

Where does it get the information that 1.06M transactions are on disk...I want to zero that, since I know replication is actually caught up and working fine now

Ron

Re: Incomming - on disk; where does this information actually exist

Dana_Baldwin — Wed, 07 Jun 2023 17:20:03 GMT

Hi @RonFusionHSLLC

This indicates that a large or long running transaction is pending commit on the source and Replicate has cached the information on disk so you don't run out of memory - the information comes from reading the source transaction log.

Does the value never fluctuate/change?

Thanks,

Dana

Re: Incomming - on disk; where does this information actually exist

Heinvandenheuvel — Thu, 08 Jun 2023 03:59:39 GMT

Hmm, if you somehow know that Replication is 'all caught up' then the easiest clean-up would be a stop followed by an as-immediate-as-possible run advanced - start by timestamp <just before stop> tables are already loaded.

Hein.

Re: Incomming - on disk; where does this information actually exist

RonFusionHSLLC — Thu, 08 Jun 2023 13:41:45 GMT

It only fluctuates when new records are introduced and replicated...once the new record activity for the day is done, it returns to the same number every day

Re: Incomming - on disk; where does this information actually exist

RonFusionHSLLC — Thu, 08 Jun 2023 13:42:54 GMT

Where is this data stored on disk, I would like to take a look at it.

Re: Incomming - on disk; where does this information actually exist

Steve_Nguyen — Thu, 08 Jun 2023 13:49:36 GMT

normally the on this would be in the sorter folder , more info:

https://community.qlik.com/t5/Official-Support-Articles/Replicate-Sorter-Files/ta-p/1872860

Re: Incomming - on disk; where does this information actually exist

Barb_Fill21 — Thu, 08 Jun 2023 14:58:29 GMT

H @RonFusionHSLLC

As everyone has suggested the transactions are stored on disk. For Example C:\Program Files\Attunity\Replicate\data\tasks\my_task_name\sorter\

The data is not viewable if you try to look at it. You cannot edit it.

Hope this helps.

Barb

Re: Incomming - on disk; where does this information actually exist

RonFusionHSLLC — Thu, 08 Jun 2023 15:42:22 GMT

I see...I'm going with Hein on this one (as usual - good answer).

I'm reconciling it this way...I only load the tables in this replication once a day, it all sits idle the rest of the time and I'm sure I have all the records from source. I'll just stopped and started using near current time as in essence anytime before the next run (at 1am tomorrow) is 'just before loading'.

That cleared out the sorter folder and the counter now shows 0...perfect

Thanks.

ron