Scenario:
Data is being loaded from Azure SQL Managed Instance (MS-CDC) to Kafka using Qlik Replication. Whenever connectivity issues occur, the task is run using Advanced Run Options (Point in Time).
Identical records are loading (Duplicate Records) multiple times. To track data integrity, the header AR_H_COMMIT_TIMESTAMP was used and the AR_H_COMMIT_TIMESTAMP is higher on the 1st committed than the second one.
When restarting the task back in time, the header information for specific changes is different every time the task is run. For example, when a change is captured the header_timestamp may be '2023-02-22 18:58:56.221000'. If the task is resumed back in time before the change the header_timestamp may be '2023-02-22 18:58:56.314000' for the exact same change.
All of the header column values are different.
Resolution
The header timestamp is not an exact value on the change.
In the SQL Server, the transaction timestamp is approximate and based on the last commit that happened in the database. Because of the way that changes are captured in MS-CDC, the header information is not part of the change record and is read from a separate table and since it is not exact you can end up with different header values.
Environment
Qlik Replicate
