QR: Log stream vs header__timestamp

[Jon_Donker]

Hi,

Just need a conformation of header__timestamp in regards to reading from the Log stream task:

Is the header__timestamp in the S3 json file the timestamp of the change on the source? 

Or the time the change was written to the log stream?

[Alan_Wang]

Hi @Jon_Donker 

Header timestamp is the timestamp of the Change operation, which is represented as the local time on the Replicate Server. Header commit timestamp is the source database commit timestamp, according to Replicate Server time. For an extensive explanation of all our header variables, please refer to the following link:

https://help.qlik.com/en-US/replicate/May2022/Content/Global_Common/Content/SharedEMReplicate/Custom...

[Jon_Donker]

Thanks Alan.

So those values are not impacted by the fact that the data is reading from a Log stream?  They still relate to the source DB2 database?

[Alan_Wang]

Header timestamp is when the change is applied for the child log stream task. No relation to the DB2 timestamps. Header commit timestamp is the one that logs DB2 commit timestamps.

[Jon_Donker]

Hi - just coming back to this problem.

The downstream users would like the "Header DB Commit Timestamp".

When we shift to log stream; is it possible to overwrite the header__timestamp with H_DB_COMMIT_TIMESTAMP so they don't have to modify their ingestion patterns to use a new field?

[Alan_Wang]

Change Tables have some prebuilt headers that can be turned on or off in task settings.

Change Processing -> Stored Changes Settings

Headers that do not come prebuilt into the table will need to be added as a new column through transformations. You will use the expression builder to add the header in as the value. You can name the column anything you like.

$AR_H_DB_COMMIT_TIMESTAMP

These changes will need a reload to rebuild the table.