topic Re: Defining Custom control topics for Kafka as target endpoints in Qlik Replicate

Defining Custom control topics for Kafka as target endpoints

adbdkb — Tue, 12 Jul 2022 13:19:14 GMT

@SwathiPulagam has helped answer the question I had about custom control topics for Kafka Target endpoint in this question - https://community.qlik.com/t5/Qlik-Replicate/Control-topics-How-to-find-actual-updates/m-p/1953555

When I tried to define those and tried to import task, it would error out. I tried with the names as below as well as using smaller in length names - similar to what Swathi had given example of, like this

"status_table_settings": {
  "table_name": "attrep_status_temp",
},
"status_table_enabled": true,
"suspended_tables_table_settings": {
  "table_name": "attrep_suspended_tables_temp"
},
"suspended_tables_table_enabled": true,
"history_table_settings": {
  "table_name": "attrep_history_temp"
},
"history_table_enabled": true,
"exception_table_settings": {
  "table_name": "attrep_apply_exceptions_temp"
},

"status_table_settings": {
  "table_name": "attrep_status_adb_sysname_tblnm_task1",
},
"status_table_enabled": true,
"suspended_tables_table_settings": {
  "table_name": "attrep_suspended_tables_adb_sysname_tblnm_task1"
},
"suspended_tables_table_enabled": true,
"history_table_settings": {
  "table_name": "attrep_history_adb_sysname_tblnm_task1"
},
"history_table_enabled": true,
"exception_table_settings": {
  "table_name": "attrep_apply_exceptions_adb_sysname_tblnm_task1"
},

Both of these give the error when importing.

This is what I have but I get an error when importing the task.

The error I get is

SYS-E-HTTPFAIL, Cannot import repository from the file /opt/attunity/replicate/data/tmp/220711154636.

If I remove the custom topic names, I am able to import the task.

What are the considerations when using custom topic names in the json?

So, wondering, what are the rules that need to be followed for these topic names, like,

1. Must they start with attrep_
2. Must the name be all lower case
3. Is there a limit on the length of the topic name

If you can help with identifying the case of the issue and provide some guidance on the names, that will be useful

Thanks

Re: Defining Custom control topics for Kafka as target endpoints

SwathiPulagam — Tue, 12 Jul 2022 13:24:53 GMT

@adbdkb you left a comma after the status table name. Please remove the below comma which I highlighted.

 "table_name": "attrep_status_adb_sysname_tblnm_task1",

Thanks,

Swathi

Re: Defining Custom control topics for Kafka as target endpoints

adbdkb — Tue, 12 Jul 2022 13:37:18 GMT

Thanks @SwathiPulagam - that worked. Appreciate your help.

Re: Defining Custom control topics for Kafka as target endpoints

adbdkb — Wed, 13 Jul 2022 09:40:50 GMT

Hi @SwathiPulagam - I corrected the json, and imported and started the task. The task has been running, but I don't see any messages in my custom control topics.

Is there any other property that needs to be changed? Here also, it still shows the default topics names and I am not able to modify those

Thanks

Re: Defining Custom control topics for Kafka as target endpoints

adbdkb — Thu, 14 Jul 2022 13:55:39 GMT

Any suggestion on the above issue? What do I need to look into?

Thanks

Re: Defining Custom control topics for Kafka as target endpoints

SwathiPulagam — Thu, 14 Jul 2022 13:57:17 GMT

Hi @adbdkb ,

In UI it will show the default topic names but messages should go to custom topics as long as JSON values are updated.

Thanks,

Swathi

Re: Defining Custom control topics for Kafka as target endpoints

adbdkb — Thu, 14 Jul 2022 14:02:40 GMT

Actually, they are not going to the custom topics. The json is as below, with the comma removed

"status_table_settings": {
  "table_name": "attrep_status_adb_sysname_tblnm_task1"
},
"status_table_enabled": true,
"suspended_tables_table_settings": {
  "table_name": "attrep_suspended_tables_adb_sysname_tblnm_task1"
},
"suspended_tables_table_enabled": true,
"history_table_settings": {
  "table_name": "attrep_history_adb_sysname_tblnm_task1"
},
"history_table_enabled": true,
"exception_table_settings": {
  "table_name": "attrep_apply_exceptions_adb_sysname_tblnm_task1"
},

But I am not seeing messages in those topics, that is why I looked into the UI? Is there any other place I need to check?

Thanks