LogStream Staging Stream Position

[Anonymous]

Hello,

We are currently testing a number of replication solutions using log stream and a question has arisen in our work around understanding what the stream position indicators denote in the log files.  We are replicating from Oracle.

Here is an example excerpt:

Start source from stream position 500;637477859264524360;20210111084947867861 and id 475378. Confirmed target id is 588049, next target id is 588053, last source timestamp 1612189124435211

So far, I have been able to deduct that 500 reflects the LogStream file that is being processed, because there are corresponding numbers in the source capture task that look the same.  So as each rollover occurs this number increases.

The far right number of the stream position appears to be some kind of extrapolation of the date (2021XX...) 
I do not know what the middle number is.  Perhaps the audit sequence?

Last source timestamp looks close to being the Oracle SCN but I am not yet certain.

It would be helpful to understand this information; it does not appear to be documented.

A similar document was written to this that explains how to parse the stream position when reading directly from Oracle.  I'm looking for a similar breakdown of this stream position information.

[Bill_Steinagle · Accepted solution]

Team,

Below is the format to get the SCN for the Oracle Binary Reader and Log Miner with Replicate.

Binary Reader Stream Position

0075.1fafdbb9.00000006.0000.00.0000:116755.160175.16

• 0075.1fafdbb9 – SCN (6 bytes before 5.2, 8 bytes from 5.2)
• 00000006 - thread

• 0000 - serial number inside current SCN
• 00 -  serial of current vector inside REDO event
• 0000 - serial number  in the multiple  insert event
• 116755 - REDO LOG SEQ
• 160175 - Block Number in REDO log
• 16 - not used

Log miner stream position

— Example:
0749A16B2C830100000000169AB40002850E00100000
6 bytes -  SCN (6 bytes before 5.2, 8 bytes from 5.2) ‘
1 bytes – thread id
3 bytes – transaction sequencer
4 bytes – rbasqn (file sequence number)
4 bytes – rbablk (block index)
2 bytes – rbabyte (offset in block)
2 bytes – context sequence
last 6 last bytes -  start SCN of the previous log miner session (6 bytes before 5.2, 8 bytes from 5.2) 

Regards,

Qlik Support

[Heinvandenheuvel · Accepted solution]

Hi @Bill_Steinagle  ! You woke up an old topic.

I'll answer one more question from that older topic:

>> Last source timestamp looks close to being the Oracle SCN but I am not yet certain.

Noop. It is a timestamp in Unix Epoch format.

Verify with https://www.epochconverter.com/

If this is your first time visiting that site, be sure to scroll down for lots of programming hints.

1612189124435211 - GMT: Monday, February 1, 2021 14:18:44.435

Hein.

[Michael_Litz]

Hi Team,

I wanted to post this link to an article about starting logstream tasks with that last timestamp value:

https://community.qlik.com/t5/Knowledge/Starting-Log-stream-task-with-timestamp/ta-p/1879839

Michael Litz

[lyka]

Hello,

Did any of the suggested solution work?

Thanks

Lyka