Qlik Compose Delta Lake tables - removing hdr__ columns from the target table

[suraju]

Hi Team, we are using Qlik Replicate & Qlik Compose to replicate the data from Oracle to Databricks Delta lake, All the table that are available in the Databricks gets loaded without any issues,

we have created a global variable in Qlik Replicate to add two new columns (last_action_date --> datetime('now')) (Action_Flg --> operation_indicator('D', 'U', 'I')) to query the table for new inserts/updates/deletes and the time by when the record gets loaded,

hence, we would like to remove the below listed audit columns from appearing in all the target tables in Databricks Deltalake tables, please could you share the steps that need to be performed to remove the below listed column in Qlik Compose

will there be any impact from removing these audit columns from the target Databricks Deltalake tables?

hdr__created_date

hdr__modified_batch

hdr__oper

hdr__ts 

Qlik Compose Qlik Replicate 

[TimGarrod · Accepted solution]

Header columns are added by Compose to support operational understanding of the data as well as specific processing steps that are taken.  These cannot be removed from the target delta tables / views.

[suraju]

Thanks @TimGarrod