Replication inconsistency

[pmaclell]

Today we ran into a situation in a customer test account where some tables were being replicated to the ODS while others were not. We confirmed that all relevant tables were included in the log stream task they just weren't all replicating. We turned on verbose logging and didn't find any errors or other meaningful information as to why this was happening. After reloading the Source to Log Stream file and then doing the same with the Log Stream to ODS  it resolved some of the tables that weren't being replicated, but not all. Has anyone seen this behavior before and know what the root cause may be?

Thanks, Paul

[john_wang]

Hello @pmaclell ,

Thank you for you opening the article. It's hard to tell the RCA this time, could you please open a support ticket with below information: 1) The two tasks Diag Packages  (with the verbose logging files you had); 2) the 'problematic' table(s) creation DDL (include PK and/or Unique Index etc information). Support team will gladly help on the issue.

Regards,

John.

[SwathiPulagam]

Hi @pmaclell ,

Mostly it could be related to metadata refresh issues. Try to create a new task with those tables and see how the data is loading.

Thanks,

Swathi

[Bill_Steinagle]

Hi @pmaclell ,

Thank you for the post to the QDI forums. As noted by Swathi there could be some issue with the Log Stream or Replicate Task reading from this LS Source. If the creating of a new Replicate Task does not resolve the issue you can setup a Direct Task pointing to the Source Endpoint the Log Stream reads from to your Target as this will confirm there is an issue with the Log Stream Staging Task (metadata) issue. 

Thanks!

Bill

[pmaclell · Accepted solution]

Thank you for your replies! We've been able to get the environment stable and has been for the past week. What seemed to resolve our issue was reloading the OLTP to LS task taking a date 2 days prior(retention setting). Allowing it to run for several minutes before reloading the LS to ODS task. Within 24 hrs all tables were being replicated to with little latency and consistent throughput.