Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi
We have some questions all around the SCN (Oracle) and LSN (Postgres) attributes within Qlik.
Attributes in focus
Observation for Postgres as a source
Questions
As the SCN (Oracle) looks different also in the above-mentioned attributes, could you please also answer the questions for Oracle as a source?
We are only using Postgres (14.x) and Oracle as a source, and only Postgres as a target.
Thanks a lot
Kind Regards, Roman
Dear @data_integration ,
That's interesting question. Sorry, it is unanswered since last 45 hours on the community.
Let me try to answer your questions as best as I can doo.
The format of the LSN Qlik stores in the above-mentioned attributes is a hexadecimal representation of the log sequence number, which is a unique identifier for each change record in the source database. The LSN consists of two parts:
1 Log file ID and
2 Byte offset within the file.
For example, 00000049/55E5C768 means log file 49, byte offset 55E5C768.
The parts after the first dot are additional information that Qlik adds to the LSN to indicate the transaction ID and the commit LSN. For example, .5.00000049/55E5E4F8 means transaction ID 5, commit LSN 00000049/55E5E4F8. This information is used by Qlik to track the transaction boundaries and ensure data consistency.
The reason for those parts is to support Qlik’s CDC (change data capture) technology, which enables real-time data ingestion and replication without affecting the performance of source systems. Qlik uses the LSN to identify the changes that need to be replicated and to resume processing from the last checkpoint in case of interruptions.
The format of the LSN is not configurable, as it is determined by the source database system. However, you can use the Advanced Run Options to restart a Qlik replicate task from a specific LSN.
The SCN (system change number) is the equivalent of the LSN for Oracle sources. It is a decimal number that increases monotonically with every database change. The format of the SCN Qlik stores in the above-mentioned attributes is the same as the source database, except that it is prefixed with a zero. For example, 0.123456789 means SCN 123456789.
__________________________________________________________________________
If you found our response useful, we kindly ask you to click on “Accept as Solution”. This will aid other users in finding the answer more effortlessly.
Thank you! 😊
Best Regards,
Deepak
😊
Dear @data_integration ,
That's interesting question. Sorry, it is unanswered since last 45 hours on the community.
Let me try to answer your questions as best as I can doo.
The format of the LSN Qlik stores in the above-mentioned attributes is a hexadecimal representation of the log sequence number, which is a unique identifier for each change record in the source database. The LSN consists of two parts:
1 Log file ID and
2 Byte offset within the file.
For example, 00000049/55E5C768 means log file 49, byte offset 55E5C768.
The parts after the first dot are additional information that Qlik adds to the LSN to indicate the transaction ID and the commit LSN. For example, .5.00000049/55E5E4F8 means transaction ID 5, commit LSN 00000049/55E5E4F8. This information is used by Qlik to track the transaction boundaries and ensure data consistency.
The reason for those parts is to support Qlik’s CDC (change data capture) technology, which enables real-time data ingestion and replication without affecting the performance of source systems. Qlik uses the LSN to identify the changes that need to be replicated and to resume processing from the last checkpoint in case of interruptions.
The format of the LSN is not configurable, as it is determined by the source database system. However, you can use the Advanced Run Options to restart a Qlik replicate task from a specific LSN.
The SCN (system change number) is the equivalent of the LSN for Oracle sources. It is a decimal number that increases monotonically with every database change. The format of the SCN Qlik stores in the above-mentioned attributes is the same as the source database, except that it is prefixed with a zero. For example, 0.123456789 means SCN 123456789.
__________________________________________________________________________
If you found our response useful, we kindly ask you to click on “Accept as Solution”. This will aid other users in finding the answer more effortlessly.
Thank you! 😊
Best Regards,
Deepak
😊