Qlik Replicate - Duplicated record and replication via multiple tasks

[FabioTavares]

I have a tasks that normally is always running and replicating data from tables. At certain time this task failed and we only detected after the maximum period for cdc retention. I then created a different task when I fully loaded some of the tables, executed it and started the first one again

In one such table I detected some records were duplicated. The table in question has no PK defined in the source so I the duplication is due to that + the fact that i used 2 different tasks (no PK I would guess, no merge ability).

What would be the correct way to avoid the situations?

[john_wang · Accepted solution]

Hello @FabioTavares ,

There are two approaches to reload tables in a task:

1. Reload the entire task
   This will reload all tables in the task.

2. Reload individual table(s)
   Only the selected table(s) will be reloaded, while the remaining tables will stay in CDC mode, for example:

Note:
If you prefer to use two separate tasks - one for reloading tables and another for CDC - you should set the preparation option to:

“TRUNCATE before loading”

Regards,

John.

[DesmondWOO · Accepted solution]

Hi @FabioTavares ,

Since Qlik Replicate may revert to an earlier position, e.g. error, to ensure all changes are applied to the target, duplicate records can occur if the target does not have a primary key or unique index, or primary key/unique index constraint is not enforced.

Regards,
Desmond
Desmond