Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Qlik and ServiceNow Partner to Bring Trusted Enterprise Context into AI-Powered Workflows. Learn More!
cancel
Showing results for 
Search instead for 
Did you mean: 
MoeE
Partner - Specialist
Partner - Specialist

Resuming a task from Last source timestamp, last SCN, or last LSN

Hi,

I have a question regarding resuming a task from the point it was last processing changes.

I am aware that this can be done by acquiring the Last source timestamp from a Replicate task when it was last stopped (article for reference). Other sources (article for reference) also show that this can be done by SCN or even LSN.

MoeE_0-1765934338482.png

(Screenshot from log file after a task was ended)

These are my assumptions on this method - This method allows you to resume a task from the best point in order to not miss any changes that occurred, but also to not send any duplicate changes to the target. Am I correct in this assumption?

Q1) - Is there any possibility of missing changes using this method? If so, I'm curious as to how exactly this would cause missing changes. E.g The mechanics of how QR works regarding this.

Q2) Is there any possibility of duplicates using this method? If so, I'm also curious as to how exactly this would cause duplicates. E.g The mechanics of how QR works regarding this.

  • Points regarding Q2
    • I have been told in the past that QR will resume a task from the earliest uncommitted transaction. I can't verify the source of this information/find evidence, but if this is true then I can see when duplicates would occur. The scenario - An earlier transaction runs for a long time but is not committed. Then subsequent transactions are committed whilst the earlier transaction still hasn't been committed, then the task is stopped before this transaction is committed. Therefore, when the task is resumed, it starts from the earliest uncommitted transaction thus replaying some transactions. 
    • However, I am also aware that a Replicate task will keep applying transactions to the target endpoint for up to 30 minutes after stopping (reference article here). So I assume that this is an unlikely cause of duplicates, but can still happen if transactions aren't applied within 30 minutes. Are there any other scenarios when duplicates would occur?

Help with these questions is appreciated as always. Thank you.

Kind regards,

Mohammed

 

 

 

 

Labels (1)
1 Reply
john_wang
Support
Support

Hello Mohammed, @MoeE 

Good morning! We previously discussed this topic (please refer to the linked article). Kindly review my earlier reply, where we explained the backend mechanism involved during a task restart.

Hope this helps.

John. 

Help users find answers! Do not forget to mark a solution that worked for you! If already marked, give it a thumbs up!