Option to Add Supplemental Logging for All columns on Database tables
Within Qlik Replicate & Enterprise Manager you have an option to add supplemental logging to a source end point. The supplemental logging that is generated by the application is specific to the make up of the table. If the table contains a PK then only the PK and changed fields are logged during an update. If the table has a UK then only the fields of the UK and changed fields are logged. If there is no UK/PK then all columns are logged. We would like an option to override this behavior and log all columns for all types of tables.
Here is why this is important. The replicate product has an option to create change tables as part of the integration. The change tables can be leveraged for downstream processing to understand what has changed and only process those changes. Driving off the key may not be what downstream applications use to determine change. In our example a table is keyed by PO number but the downstream change is driven off the product number. The product number, not being a key or changed, does not appear in the change table.
Adding supplemental logging can be done manually but needs to be done every time a new table is added. This option would eliminate the need for a manual work around.
If supplemental logging is turned on for the PK columns, in case of delete in source table, Qlik is capturing the deleted row with CDC trans type=Delete as per settings. But with that row, Qlik is replicating another row with all NULL values in other columns and only value in the PK column to target Snowflake. And for this extra NULL values row, downstream application having problem to process.
We need a Option to Turn on the Supplemental Logging for All Columns in the Source DB tables.
NOTE: Upon clicking this link 2 tabs may open - please feel free to close the one with a login page. If you only see 1 tab with the login page, please try clicking this link first: Authenticate me! then try the link above again. Ensure pop-up blocker is off.
