idea Re: Compose for Data Lakes behavior when the source primary key value change - Status changed to: Open - Collecting Feedback in Suggest an Idea

Compose for Data Lakes behavior when the source primary key value change

sanlamZA — Wed, 05 May 2021 12:40:33 GMT

Our Cloudera DataLake holds data from various sources with no insight or control on the source data changes.

Currently when changing a Primary Key value in the source record it will cause a new record to be inserted in the storage table.

Due to this behavior our BI environment displays an huge amount of duplicate values when selecting from the storage base table or the current view.

We have a need for this behavior to change. A possible solution maybe to have a header deleted value for the before image.

Thanks

Janine

Re: Compose for Data Lakes behavior when the source primary key value change

Abrie_M — Wed, 29 Sep 2021 11:30:44 GMT

Good day

Any update on this request?

Thanx

Abrie Marais

Re: Compose for Data Lakes behavior when the source primary key value change - Status changed to: Open - Collecting Feedback

Tzachi_Nissim — Tue, 16 Nov 2021 11:12:00 GMT

Thank you. We are looking at addressing this in Replicate by turning the BEFOREIMAGE+UPDATE into a DELETE+INSERT when a PK change is involved. This is a relatively high priority item. I am putting this into "Collecting feedback" until we formally have this in the plan.

Re: Compose for Data Lakes behavior when the source primary key value change - Status changed to: Delivered

Meghann_MacDonald — Mon, 18 Sep 2023 15:05:59 GMT

This feature has been delivered in Qlik Replicate November 2022. Thank you for your contributions! Qlik Replicate can now deliver a PK update as a DELETE (prior key) and INSERT (new key) pair. Review the configuration - https://help.qlik.com/en-US/replicate/May2023/Content/Global_Common/Content/SharedEMReplicate/Customize%20Tasks/tasks_applychangtunestab.htm#anchor-4