Skip to main content

Suggest an Idea

Vote for your favorite Qlik product ideas and add your own suggestions.

Announcements
Action-Packed Learning Awaits! QlikWorld 2023. April 17 - 20 in Las Vegas: REGISTER NOW

Add column in attrep_status to specify last processed timestamp from source system

Prabodh
Creator II
Creator II

Add column in attrep_status to specify last processed timestamp from source system

The attrep_status table provides the replication status at a task level. This table is updated every minute.

This table contains 'SOURCE_TIMESTAMP_APPLIED' column which tells the timestamp until which the data is applied to the target. There are a few other columns that provide additional information.

However, we are looking to build a monitoring process to determine if the task has processed all the data for a given day. If we see SOURCE_TIMESTAMP_APPLIED value > midnight, we can confidently say that the task has processed all the data for the prior day for the tables it is replicating.

However, this is not a foolproof solution. If the tables being replicated within the task are not high velocity, then the SOURCE_TIMESTAMP_APPLIED corresponds to the time when data was committed to target for the task.

E.g. if the tables within the task are only updated in CST business time, we may not see any change to SOURCE_TIMESTAMP_APPLIED post 5:00 PM CT. Our monitoring process cannot determine if the data for these tables was processes for yesterday, as it has no way to know that there was no data in source after 5 PM.

 

I would like to propose an enhancement to add 'SOURCE_TIMESTAMP_PROCESSED' or equivalent column. This should be the timestamp value of the most recent transaction from source that was completely processed. This transaction may or may not produce any data for the tables in that task.