topic How does Compose for DW deals with deleted rows in the source db ? in Qlik Compose

How does Compose for DW deals with deleted rows in the source db ?

ThiebaudS — Fri, 14 Mar 2025 20:49:07 GMT

Hi everyone,

I don't really understand how Compose for DW deals with deleted rows in the source database/landing zone.

If I delete some rows in my source database, the Qlik Replicate task will delete the rows in my landing zone. This is OK.

After that, if I run the "CDC ETL Set" in Compose, it will not detect any changes and the deleted rows will still be in my data warehouse tables, and therefore of course, still be in my datamart.

Can someone help me understand this behaviour ?

Thanks a lot.

Best regards,

Thiebaud

Re: How does Compose for DW deals with deleted rows in the source db ?

Madhavi_Konda — Wed, 03 Jun 2020 19:00:22 GMT

Hi Thiebaud,
Could you please answer the following?
What is your Compose for Datawarehouses version?
What are Source and target databases for this behavior?

Thanks,
Madhavi

Re: How does Compose for DW deals with deleted rows in the source db ?

ThiebaudS — Wed, 03 Jun 2020 20:20:56 GMT

Hi @Madhavi_Konda

Thank you for your answer.

I'm using Compose 6.6.0.104, with an Oracle database as source, and Snowflake as target.

Best regards,

Thiebaud

Re: How does Compose for DW deals with deleted rows in the source db ?

TimGarrod — Wed, 03 Jun 2020 20:28:05 GMT

Thiebud - you are correct that Compose does not automatically handle delete operations currently.

Since Compose is built to load a data warehouse and typically data warehouse don't delete data.

However there are of course reasons for performing deletes and if you have that requirement, there is a methodology to support soft deletes and to then determine if you want to perform a hard delete (be aware that hard deletes can obviously cause data integrity issues in the data vault if you are not careful, and a best practice is to configure Replicate and Compose to handle soft deletes.)

We are in fact about to publish a how-to-guide soft deletes in this forum. I can provide you with an advanced copy while its finalized. Will DM you for your email address

Re: How does Compose for DW deals with deleted rows in the source db ?

ThiebaudS — Thu, 04 Jun 2020 06:36:30 GMT

Thank you @TimGarrod

Re: How does Compose for DW deals with deleted rows in the source db ?

barbm — Thu, 30 Jul 2020 04:04:00 GMT

If you add a soft delete field using Global Transformations in Replicate (something like this: LandingIsDeleted = COALESCE(operation_indicator(1, 0, 0),0) ), then the record will be flagged as 1 for deleted in the Landing database, which is the source for DW. In Compose for DW, include that delete flag field for each entity and it will update accordingly.