Skip to main content

Suggest an Idea

Vote for your favorite Qlik product ideas and add your own suggestions.

Announcements
Qlik Connect 2024! Seize endless possibilities! LEARN MORE

Replicate - Add millisecond precision to Commit Timestamp (header column) for MongoDB Source

Adam_Brillhart
Contributor II
Contributor II

Replicate - Add millisecond precision to Commit Timestamp (header column) for MongoDB Source

We are using Replicate with a MongoDB Source and an Oracle Target.

We would really like the AR_H_COMMIT_TIMESTAMP/AR_H_TIMESTAMP header columns to be populated with a timestamp value that has greater precision than a second.

Other sources populate those columns down to the millisecond, but it only goes down to the second with a MongoDB Source.

The MongoDB Oplog does contain a commit timestamp field that has the greater precision, but the current one that is being pulled from the Oplog only goes down to the second.

5 Comments
Shelley_Brennan
Former Employee
Former Employee

Replicate relies on the official MongoDB Change Streams API which only reports on the commit timestamp down to the second, not millisecond.  It may be stored in the Oplog in milliseconds but Replicate is not designed to read directly from Oplog.  I would suggest opening a case with MongoDB support to see if they can expose the timestamp in milliseconds via the API at which point Replicate could report on the data you are looking for.  And we will raise this with MongoDB support as well on our side.

Status changed to: Open - Collecting Feedback
Adam_Brillhart
Contributor II
Contributor II

@Shelley_Brennan 

Hi Shelley,

I was wondering if you could tell me the specific API call from from the MongoDB Change Streams API that is being used by Replicate to get that commit timestamp value?
I'm not sure if you have raised a MongoDB Support ticket yet, but we will use that info to raise a MongoDB Support ticket as well.

Thanks.

Shelley_Brennan
Former Employee
Former Employee

Yes, we rely on https://docs.mongodb.com/manual/reference/change-events/ - specifically clusterTime.

 

Meghann_MacDonald

From now on, please track this idea from the Ideation portal. 

Link to new idea

Meghann

NOTE: Upon clicking this link 2 tabs may open - please feel free to close the one with a login page. If you only see 1 tab with the login page, please try clicking this link first: Authenticate me! then try the link above again. Ensure pop-up blocker is off.

Ideation
Explorer II
Explorer II
 
Status changed to: Closed - Archived