CDC (Change Data Capture)

Vikash2024 — Wed, 19 Mar 2025 21:46:28 GMT

What is CDC ? Can anyone explain with an example?
And suppose i have data from 1st Feb 2025 to 28th Feb 2025
I need to implement CDC on daily basis like from 1st March 2025.
How to do that?

Re: CDC (Change Data Capture)

diegozecchini — Tue, 18 Mar 2025 07:44:34 GMT

Hi!
Qlik CDC can be implemented using Qlik Replicate or through Qlik Sense/QlikView with incremental load.

You can track changes using database logs (e.g., MySQL binlog, SQL Server CDC, PostgreSQL WAL), Timestamps (e.g., UpdatedAt column), triggers on tables (if database supports it)

Before starting CDC, ensure you have loaded the full dataset until 28th Feb 2025.
SELECT * FROM Orders WHERE Timestamp < '2025-03-01';

From 1st March 2025, you need to load only the new and updated records daily.

Example Query for Daily CDC (for 1st March 2025)
SELECT * FROM Orders WHERE Timestamp >= '2025-03-01' AND Timestamp < '2025-03-02';

If using Qlik, set up an Incremental Load Script:
LET vLastLoadDate = '2025-02-28';

Orders:
LOAD OrderID, Customer, Amount, Status, Timestamp
FROM Database
WHERE Timestamp > '$(vLastLoadDate)';

LET vLastLoadDate = Today(); // Update last processed date

Schedule the CDC process daily in Qlik Sense/QlikView, if using Qlik Replicate, configure continuous replication from source to destination.

Re: CDC (Change Data Capture)

Vikash2024 — Tue, 18 Mar 2025 11:32:13 GMT

Thanks for sharing it will really helpful for me.

Re: CDC (Change Data Capture)

diegozecchini — Wed, 19 Mar 2025 15:40:45 GMT

Hi! I am glad it helped, have a good day

topic Re: CDC (Change Data Capture) in Data Movement & Streaming

CDC (Change Data Capture)

Re: CDC (Change Data Capture)

Re: CDC (Change Data Capture)

Re: CDC (Change Data Capture)