Resuming task using source sequence number

[MoeyE]

Hi,

I have a general question I'm wondering about. I saw in this article that you can resume or restart a task using the sequence number shown in the logs when a task stops. I understand that this is available only for certain endpoints.

https://community.qlik.com/t5/Official-Support-Articles/How-to-Start-Replicate-task-from-timestamp-o...

So this appears to be the best "timestamp" to restart/resume a task without data loss or duplicates. However I am wondering if there is a possibility of the scenario where there could have been a long running uncommitted transaction that would be skipped if we restart/resume using this "timestamp". Is this possible or not? Please help my understanding.

Regards,

Mohammed

[DesmondWOO]

Hi @MoeyE ,

Yes, it is possible that some transactions may be lost. For example,

15:00 Begin Transaction
15:02 Insert Record 
15:04 Commit

Let's say you resume task at 15:01. Replicate ignores the inserted record because it can't find the corresponding BEGIN transaction.

Regards,
Desmond

[john_wang]

Hello @MoeyE ,

Besides @DesmondWOO comments, I'd like to separate the 2 startup modes of

>> restart/resume using this "timestamp"

1- there is significant difference between RESUME and restart from "timestamp"

    RESUME: Qlik Replicate will load the .TSWP files from disk to memory and continue processing the 'in doubt' transactions in Desmond's sample. This is because Qlik Replicate will store the 'uncompleted' transaction into .TSWP files in previous task stop.

    However, if the task is restarted from timestamp, then no .TSWP file will be handled and the 'in doubt' transactions will be ignored.

2- Desmond's data missing scenario occurs for some DB only, eg Oracle source.

Hope this helps.

John.

[MoeyE]

Hi @john_wang and @DesmondWOO ,

Thank you  for the help.

Regards,

Mohammed

[john_wang]

Thank you for your great support Mohammed! @MoeyE