Qlik Community

Suggest an Idea

Vote for your favorite Qlik product ideas and add your own suggestions.

Announcements
QlikWorld 2022, LIVE in Denver CO., May 16-19, 2022. REGISTER NOW TO RECEIVE EARLY BIRD PRICING

Replicate - Add millisecond precision to Commit Timestamp (header column) for MongoDB Source

Adam_Brillhart
Contributor II
Contributor II

Replicate - Add millisecond precision to Commit Timestamp (header column) for MongoDB Source

We are using Replicate with a MongoDB Source and an Oracle Target.

We would really like the AR_H_COMMIT_TIMESTAMP/AR_H_TIMESTAMP header columns to be populated with a timestamp value that has greater precision than a second.

Other sources populate those columns down to the millisecond, but it only goes down to the second with a MongoDB Source.

The MongoDB Oplog does contain a commit timestamp field that has the greater precision, but the current one that is being pulled from the Oplog only goes down to the second.

3 Comments
Shelley_Brennan
Employee
Employee

Replicate relies on the official MongoDB Change Streams API which only reports on the commit timestamp down to the second, not millisecond.  It may be stored in the Oplog in milliseconds but Replicate is not designed to read directly from Oplog.  I would suggest opening a case with MongoDB support to see if they can expose the timestamp in milliseconds via the API at which point Replicate could report on the data you are looking for.  And we will raise this with MongoDB support as well on our side.

Status changed to: Open - Collecting Feedback
Adam_Brillhart
Contributor II
Contributor II

@Shelley_Brennan 

Hi Shelley,

I was wondering if you could tell me the specific API call from from the MongoDB Change Streams API that is being used by Replicate to get that commit timestamp value?
I'm not sure if you have raised a MongoDB Support ticket yet, but we will use that info to raise a MongoDB Support ticket as well.

Thanks.

Shelley_Brennan
Employee
Employee

Yes, we rely on https://docs.mongodb.com/manual/reference/change-events/ - specifically clusterTime.