Issue while switching between DC to DR environment

[Supriya_I]

Hi,
We are using Microsoft-cdc as source and Oracle as Target in Qlik replicate. The source table have primary key and have duplicates in it.
The challenge we are facing here is: When there is a switch done between DC and DR we are facing issue.

> From the logs we got the stream value of the last processed log. Using the Epoch converter we are converting this to timestamp which is giving us millisecond information. Since we are unable to resume a task using milliseconds, it is resulting in a few duplicate that are moved to the target.
> We cannot use the UPSERT method here since, they don't want the actual duplicate from the source to be merged in the target.
> Resumed the task using LSN number from logs getting error:
      (1) " Failed to get time mapping from lsn position "----LSNNUMBER----" for start from position. The required "----LSN NUMBER----" not found. Tables must be reloaded"
      (2) Stream Component 'st_0_mssql_dr' terminated. Stream component failed a subtask 0, component             st_0_mssql_dr. Error executing source loop"

> We tried fetching the LSN number from MS SQL backend, we find that the source database LSN from backend is different from the LSN number which we are getting in the log file. Resumed the task using source backend LSN as well but got same error.
> Enabled “Store task recovery data in target database ” from change processing tuning. While performing any transformation we are getting error " Task stopped abnormally". I have attached the log file regarding same.
Thank you

Regards,
Supriya I

[john_wang]

Hello @Supriya_I ,

Welcome to Qlik Community forum and thanks for reaching out here!

>> The source table have primary key and have duplicates in it.

I guess I did not get it accurately. If the source table has Primary Key then it's impossible to have duplicate rows in source table.

Additionally, if the target table in Oracle has a Primary Key, it will automatically prevent duplicate rows from being written to the table. This is a built-in feature of the Oracle database.

From your task log file, we noticed you are facing the known issue An exception occurred!!!. The task is keeping in crash. Please upgrade to 2024.5 PR02 to fix the issue.

Hope this helps.
John.

[john_wang]

Hello @Supriya_I ,

You may download 2024.5 SP02 from below link:

https://files.qlik.com/url/qr2024050316sp02 (expires 10/31/2024)

Regards,

John.

[Supriya_I]

Hi John,

Sorry for the confusion. Source table doesn’t have primary key.

Regards,

Supriya I

[sureshkumar]

Hello @Supriya_I 

I have reproduced the SCN issue in my lab. Let's work on the case you've created.

I have explained in the case on how to use the LSN.

Regards,
Suresh