topic Re: CDC in Qlik Replicate

CDC

Vandy — Mon, 13 Mar 2023 11:52:03 GMT

Hi team,
I have the following doubts while running a CDC task (Oracle being the source database and Kafka as the target using unidirectional task)

CDC task is running and failed due to source availability issue. Can resume be done. If not, what will happen if reload is chosen.
CDC task is running, and the task is paused, after that the database is restarted. If we resume the task after the database is up, what will be the starting point for the task.
Whether QDI is storing any reference on the CDC, like till when the replicate is completed. If yes, where this is getting stored. If not, then on what basis the recovery is happening.
In the monitoring interface of the task, insert/update/delete count from source and target is displayed. Is it getting stored in the database? If yes, where it is getting stored? If no, the how this info is getting displayed?

Kindly answer to my doubts at the earliest.

Thank you!

Re: CDC

john_wang — Mon, 13 Mar 2023 13:17:27 GMT

Hello @Vandy ,

Thanks for reaching out.

In Qlik Replicate , there is stream position value on the source, usually the SCN, LSN, or POS etc which allows Replicate tasks to resume from the last broken point, it's stored in Replicate repository, or target database (configurable).

1- Yes, if a task is stopped and resumed, them it will resume from the last stream position (In Oracle it's SCN) automatically. Certainly you could startup the task from a given SCN/LSN/POS/Timestamp etc manually.

2- From last stream position, if it's available still.

3- In general Stream Position stores in Qlik Replicate Repository. Replicate utilizes SQLite DB as Repository.

4- Yes or No. Yes - the information is stored in SQLite DB however you cannot access the SQLite DB directly; No - If you want to get these values, QEM is recommended.

Hope this helps.

Regards,

John.

Re: CDC

Vandy — Tue, 14 Mar 2023 07:08:40 GMT

Hello @john_wang ,

Thank you so much for clarifying all my doubts. It was very helpful.

I also wanted to know if I can get the Table Statuses data without having Qlik Enterprise Manager.

Thank you

Re: CDC

john_wang — Tue, 14 Mar 2023 13:07:19 GMT

Hello @Vandy ,

QEM APIs are uniform among different QEM versions, it's the easiest and standard approach. Maybe it's doable in Replicate but it's some complex and it's better involve PS team to implement it.

Regards,

John.

Re: CDC

Vandy — Thu, 16 Mar 2023 06:48:00 GMT

Thank you @john_wang