We are planning to load 100's of tables from an Oracle DB into Azure SQL Server using Qlik Replicate CDC. The plan is to load data from the source system (Oracle) into our ODS layer in Azure SQL using Qlik Attunity.
We will then have an ETL process running overnight to refresh our Data Warehouse layer from the ODS layer.
Do the experts on this forum recommend stopping the "Qlik Replicate CDC process" prior to running the overnight ETL process & then starting it up again after the ETL process is complete ?
My first thought is no, do not start and stop the replication process. This may cause a build up of transactions on the source side, task latency, plus more complexity in your overall end to end process.
Does your ETL process capture the data in Azure SQL (your target) as a static or snapshot of the table or will it be impacted by incoming records?
I would test this on a smaller set of tables and a simple ETL job, to see if it can handle ongoing replication rather than stop/start pattern post job.
.png "Former Employee"){kind=icon}
