idea Option to Add Supplemental Logging for All columns on Database tables in Suggest an Idea

Option to Add Supplemental Logging for All columns on Database tables

Troy — Wed, 03 Jun 2020 17:24:50 GMT

Within Qlik Replicate & Enterprise Manager you have an option to add supplemental logging to a source end point. The supplemental logging that is generated by the application is specific to the make up of the table. If the table contains a PK then only the PK and changed fields are logged during an update. If the table has a UK then only the fields of the UK and changed fields are logged. If there is no UK/PK then all columns are logged. We would like an option to override this behavior and log all columns for all types of tables.

Here is why this is important. The replicate product has an option to create change tables as part of the integration. The change tables can be leveraged for downstream processing to understand what has changed and only process those changes. Driving off the key may not be what downstream applications use to determine change. In our example a table is keyed by PO number but the downstream change is driven off the product number. The product number, not being a key or changed, does not appear in the change table.

Adding supplemental logging can be done manually but needs to be done every time a new table is added. This option would eliminate the need for a manual work around.

Re: Option to Add Supplemental Logging for All columns on Database tables - Status changed to: Open - Collecting Feedback

Ola_Mayer — Mon, 06 Jul 2020 12:39:17 GMT

Re: Option to Add Supplemental Logging for All columns on Database tables

qlik_admin_lm — Tue, 12 Oct 2021 18:57:54 GMT

Adding to the above issue, we have the below one:

If supplemental logging is turned on for the PK columns, in case of delete in source table, Qlik is capturing the deleted row with CDC trans type=Delete as per settings. But with that row, Qlik is replicating another row with all NULL values in other columns and only value in the PK column to target Snowflake. And for this extra NULL values row, downstream application having problem to process.

We need a Option to Turn on the Supplemental Logging for All Columns in the Source DB tables.

Thanks.

Regards,

Sandip Saha

From now on, please track this idea from the Ideation por...

Meghann_MacDonald — Wed, 02 Aug 2023 15:51:42 GMT

From now on, please track this idea from the Ideation portal.

Link to new idea

Meghann

NOTE: Upon clicking this link 2 tabs may open - please feel free to close the one with a login page. If you only see 1 tab with the login page, please try clicking this link first: Authenticate me! then try the link above again. Ensure pop-up blocker is off.

Re: Option to Add Supplemental Logging for All columns on Database tables - Status changed to: Closed - Archived

Ideation — Wed, 02 Aug 2023 15:51:44 GMT