topic Re: Replication functionality in Qlik Replicate

Replication functionality

nahowe — Fri, 26 Jul 2024 18:49:53 GMT

Hi,

I need help understanding something I can't seem to find in documentation. Our replication use case is to replicate to the target everything that wasn't deleted by a specific user in the source. In the scenario that a row was deleted in source, replicated to target, but added back to source later - How does Qlik handle unwanted duplicated records in the target?

Re: Replication functionality

SachinB — Sat, 27 Jul 2024 04:58:07 GMT

Hello @nahowe ,

Thanks for contacting Qlik community forum.

If a row that was deleted and added back later, it should be treated as a new row during replication. However, if the row was previously deleted on the target as part of the CDC process (due to a DELETE DML operation), the new transaction will still be handled correctly

You can follow the below community link on order to handle the Duplicate records:

Duplicate-Data-Handling

Qlik-Replicate-Apply-conflicts

Regards,

Sachin B

Re: Replication functionality

john_wang — Sat, 27 Jul 2024 08:09:19 GMT

Hello @nahowe ,

If I understand is correctly, you are meaning:

1- You added filter in the task to filter out the DELETE operations;

2- You deleted a row in source side (the DELETE will be filtered out so the specific row exists in target table still), and then you insert the same PK row back in source side.

In this case, what you want Qlik Replicate to do? If you want Replicate UPDATE the target row if it exists, then you may set the task to UPSERT Mode. So far if the INSERT operation will be converted to UPDATE operation to make sure no unwanted duplicate records in the target side.

Certainly there are more options as blow, please check Duplicate key when applying an INSERT.

Ignore record: The task continues and the error is ignored.
Log record to the exceptions table (default): The task continues and the record is written to the exceptions table.
Suspend table: The task continues but data from the table with the error record is moved into an error state and its data is not replicated.
Stop task: The task is stopped and manual intervention is required.
Update the existing target record: The target record with the same primary key as the INSERTED source record is updated.
Allow duplicates in the target: Allows duplicate primary key records to be added to the target table. When Optimize inserts is enabled, this is the only option that is available at task level error handling.

Hope this helps.

John.