topic Re: How to make transaction id as custom key from the header data for the event in Qlik Replicate

How to make transaction id as custom key from the header data for the event

murali23krishna — Tue, 12 Dec 2023 15:27:29 GMT

We have a scenario where we do not want to use the table primary key as the key, instead, I would want to use the transaction id present in the headers, is there a way to achieve it by using transformations?

Re: How to make transaction id as custom key from the header data for the event

DesmondWOO — Wed, 13 Dec 2023 04:45:28 GMT

Hi @murali23krishna ,

Thanks for reaching Qlik community.

On the transformation page, you can add a new column with $AR_H_TRANSACTION_ID value to get the transaction ID. However,

1. There is no transaction ID for full load. It will cause skewed data to one partition.
2. A discrepancy exists between $AR_H_TRANSACTION_ID and Kafka's transactionId. For example,

$AR_H_TRANSACTION_ID : "E0FAB00100000000"
Kafka's header transactionId: "E0FAB001000000000000000000000000"

Regards,
Desmond

Re: How to make transaction id as custom key from the header data for the event

murali23krishna — Wed, 13 Dec 2023 06:22:20 GMT

Is this $AR_H_TRANSACTION_ID is unique for every transaction or it can be duplicated across?

Re: How to make transaction id as custom key from the header data for the event

DesmondWOO — Wed, 13 Dec 2023 06:38:16 GMT

Hi @murali23krishna ,

According to the user guide , the uniqueness of the Transaction ID is not guaranteed.

Regards,
Desmond

Re: How to make transaction id as custom key from the header data for the event

murali23krishna — Wed, 13 Dec 2023 06:42:10 GMT

Is there any other field from the header can be used as unique field?

Re: How to make transaction id as custom key from the header data for the event

DesmondWOO — Wed, 13 Dec 2023 07:20:25 GMT

Hi @murali23krishna ,

Which source database are you using?

Regards,
Desmond

Re: How to make transaction id as custom key from the header data for the event

murali23krishna — Wed, 13 Dec 2023 07:21:49 GMT

It is a oracle database

Re: How to make transaction id as custom key from the header data for the event

DesmondWOO — Wed, 13 Dec 2023 10:23:54 GMT

Hi @murali23krishna ,

You can try $AR_H_CHANGE_SEQ. Please be aware that resuming the task from a previous position will result in a different change sequence for the same record.

Regards,
Desmond