Format of Qlik SCN or LSN attributes

[data_integration]

Hi

We have some questions all around the SCN (Oracle) and LSN (Postgres) attributes within Qlik.

Attributes in focus

• attrep_status.source_current_position
• <changeTable>.stream_position

Observation for Postgres as a source

• The format of the LSN in the source DB and in the above-mentioned attributes is not quite the same (the format of the 2 attributes are the same)
• Qlik: 00000049/55E5C768.5.00000049/55E5E4F8
• DB: 00000049/55E5C768

Questions

• What is the format of the LSN Qlik stores in the above-mentioned attributes?
• What are the parts after the first dot? (.5.00000049/55E5E4F8)
• What is the reason for those parts (we just want to understand)?
• Is this somehow configurable what the format looks like?

As the SCN (Oracle) looks different also in the above-mentioned attributes, could you please also answer the questions for Oracle as a source?

We are only using Postgres (14.x) and Oracle as a source, and only Postgres as a target.

Thanks a  lot

Kind Regards, Roman

[deepaksahirwar · Accepted solution]

Dear @data_integration ,

That's interesting question. Sorry, it is unanswered since last 45 hours on the community.

Let me try to answer your questions as best as I can doo.

The format of the LSN Qlik stores in the above-mentioned attributes is a hexadecimal representation of the log sequence number, which is a unique identifier for each change record in the source database. The LSN consists of two parts:

1 Log file ID and

2 Byte offset within the file.

For example, 00000049/55E5C768 means log file 49, byte offset 55E5C768.

The parts after the first dot are additional information that Qlik adds to the LSN to indicate the transaction ID and the commit LSN. For example, .5.00000049/55E5E4F8 means transaction ID 5, commit LSN 00000049/55E5E4F8. This information is used by Qlik to track the transaction boundaries and ensure data consistency.

The reason for those parts is to support Qlik’s CDC (change data capture) technology, which enables real-time data ingestion and replication without affecting the performance of source systems. Qlik uses the LSN to identify the changes that need to be replicated and to resume processing from the last checkpoint in case of interruptions.

The format of the LSN is not configurable, as it is determined by the source database system. However, you can use the Advanced Run Options to restart a Qlik replicate task from a specific LSN.

The SCN (system change number) is the equivalent of the LSN for Oracle sources. It is a decimal number that increases monotonically with every database change. The format of the SCN Qlik stores in the above-mentioned attributes is the same as the source database, except that it is prefixed with a zero. For example, 0.123456789 means SCN 123456789.

__________________________________________________________________________

If you found our response useful, we kindly ask you to click on “Accept as Solution”. This will aid other users in finding the answer more effortlessly.

Thank you! 😊

Best Regards,
Deepak

 😊