Qlik Replicate Limitation and Consideration

[lqthinguyen]

Good evening 

• It is not recommended to performing database maintenance tasks during replication as doing so might result in unpredictable behavior.

TLN - Is the recommendation applied to both SOURCE and TARGET databases

• If a task fails with a recoverable error on the target while it is starting, it will not read changes from the source.

TLN - Please  clarify "while it is starting"  as in Qlik Replicate task

Thank you

[cristianj23a]

Hi Lqtinguyen.

Qlik Replicate is a data integration product that allows you to move and replicate data across various databases, data lakes, and data warehouses. The limitations and considerations you mentioned generally refer to the way Qlik Replicate interacts with the source and target databases during replication.

When it says it is not recommended to perform database maintenance tasks during replication, it's generally speaking about both the source and target databases. Any changes made to the database schema, for instance, adding or dropping tables, modifying indices, etc., while replication is ongoing could interfere with the replication process and potentially cause errors. These changes can affect both source and target databases as Qlik Replicate relies on the metadata and structure of both to replicate data accurately.

Regarding the term "while it is starting", it is referring to the period when a Qlik Replicate task begins its operation. During the initiation of a replication task, Qlik Replicate establishes connections to both the source and target databases, begins reading the source database's transaction log, and starts replicating data to the target database. If a recoverable error occurs on the target database during this initiation phase, Qlik Replicate would stop reading changes from the source database until the error on the target is fixed, and the replication task can be safely resumed. This is to ensure data consistency and integrity during the replication process.

Regarts.

[Heinvandenheuvel · Accepted solution]

>> It is not recommended to performing database maintenance tasks during replication as doing so might result in unpredictable behavior.

> TLN - Is the recommendation applied to both SOURCE and TARGET databases

The warning is mostly for the source. But at some point maintenance is going to be needed. Depending on the specific source it may work better to 'pause' (= STOP + RESUME) the Replication process over the maintenance window.

Potential issues are - extended table/object locks. New table-ids causing the need to re-scan (archived) transaction logs - high source resource usage reducing 'room' for the small, but noticeable load for Replication.

The target apply tends to be more 'robust'. All the changes are already on the Replicate server, and after a failure it can just try again.

[lqthinguyen · Accepted solution]

Good morning Heinvandenheuvel,

Thank you for more an explicit information  even though it's very logical from a DBA's perspective; basically the more information we have the better decision-making for our system  Once again thank you