topic Re: Automatically enable Change Data Feed on each table from Qlik Replicate in Qlik Replicate

Automatically enable Change Data Feed on each table from Qlik Replicate

MoeE — Thu, 28 Nov 2024 01:56:50 GMT

Hi,

A client needs to enable Change Data Feed on their target tables. Change Data Feed is a setting in Databricks. As they are using Databricks SQL Warehouse type compute, they are forced to enable it on each table manually.

I am wondering if there is an internal parameter which allows us to enable this on each table automatically from the Qlik Replicate side? Otherwise, the alternative is to run a full load, stop the task, then enable the setting on each table manually.

Help with this is much appreciated as always. If there is no solution, that's ok. Thank you!

Regards,

Mohammed

Re: Automatically enable Change Data Feed on each table from Qlik Replicate

DesmondWOO — Thu, 28 Nov 2024 05:35:33 GMT

Hi @MoeE ,

We don't have internal parameters to enable the change data feed.

According to the Databricks website, we can enable the change data feed option through the CREATE TABLE statement. I suggest you try editing the syntax file, but I haven't tested it myself.

Regarding customizing the syntax file, you can refer to this article: Qlik Replicate: How to custom target endpoint syntax.

Regards,
Desmond

Re: Automatically enable Change Data Feed on each table from Qlik Replicate

MoeE — Thu, 28 Nov 2024 05:38:36 GMT

Hi Desmond,

Thanks for the help.

Regards,

Mohammed

Re: Automatically enable Change Data Feed on each table from Qlik Replicate

MoeE — Thu, 28 Nov 2024 23:26:25 GMT

Hi Desmond,

Is it possible to edit the "query_syntax" section of the provider syntax.json file to add an additional new custom query that will execute on each table when the task is stopped and resumed? e.g add a new query called "add_change_data_feed": "alter table ..."

Can I only edit the pre-existing queries here?

Regards,

Mohammed

Re: Automatically enable Change Data Feed on each table from Qlik Replicate

DesmondWOO — Fri, 29 Nov 2024 01:30:51 GMT

Hi @MoeE ,

Based on my understanding, we cannot add a new query. Additionally, not all queries are eligible for modification.

Regards,
Desmond

Re: Automatically enable Change Data Feed on each table from Qlik Replicate

MoeE — Fri, 29 Nov 2024 01:31:46 GMT

Hi Desmond,

Ok thanks for the info.

Regards,

Mohammed