
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Change KEY values going to Kafka
We have a Qlik task with an Oracle source and a Kafka target (key columns defined). The app that updates Oracle changes data in the unique index, in the table. The problem for us, is we are using the unique key in Oracle as a primary key downstream of Kafka. The index data change is causing a Primary Key change down stream which is causing update problems (can't find the current record because the key changed). The question is, can we change just the key values in Qlik? I know there is before and after info, but we need to change the key values going to Kafka.
"key":
{"FACILITY":{"string":"BNP"},
"TICKET_NUMBER":{"string":"94054100"},
"WHSE":{"string":"1"},
"CATALOG_ID":{"string":"0073530040"},
"Q_LEVEL":{"string":"0"},
"UTC_NUMBER":{"string":" },
"QTY_TYPE":{"string":"PR"},
"LOC_ZONE":{"string":"3"},
"LOC_ROW":{"string":"D"},
"LOC_SECTION":{"string":"3A"},
"LOC_TIER":{"string":"3ZZ"},
"LOC_BIN":{"string":"ZZZ"}},
"value"
_data": { ...
_beforeData": { ...

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello @fj40wdh ,
Thanks for reaching out to Qlik Community!
I'm uncertain about the impact of changing the primary key or unique index on the downstream components of Kafka. Are you concerned about how this change might affect which partition the messages are sent to?
If yes, we have a similar discussion, Kafka Partition strategy by primary key.
Hope this helps.
John.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks!
I am not concerned about the key in Kafka. We will only be changing the value of one of the columns in the key. The partition will not be affected. Just wondering if there is a key variable like the before variable $BI__UTC_NUMBER or current value $UTC_NUMBER that be accessed to change the key value while leaving the current value as is?

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello @fj40wdh ,
Thanks for the update.
By default the Kafka message contains the record before image in "beforeData" section (see ②), however you still can use the $BI__<columnName> to get additional value of the before image (see ①), a sample of the column "name" before image and after image:
Hope this helps.
John.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks
I'm aware of the before image and after image. I'm need to know about the KEY "image" see the example in my original post. when you look at the topics in Kafka there is a KEY section, a CURRENT DATA section, and a BEFORE DATA section. All of this comes from Qlik Replicate. I know I can change the Before and After image data, I need to know if there is a way to change just the KEY data,
Note
You can change the after image and it will change the key, but I only need to change the key.
Thanks.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello @fj40wdh ,
I guess I did not get the question well. Would you please export the task and attach the json - obfuscating sensitive information as needed.
Thank you,
John.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Here's the file

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Here's a file with the record from Kafka, you can see the KEY section, then VALUE section, under the value section is the AFTER and BEFORE data.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello @fj40wdh ,
Thanks for the information. Would you please open a support ticket with source table creation DDL and we'd like to reproduce the behavior in our internal labs to understand the issue. I'd love to work with out support team to move the case forward.
thanks,
John.
