topic Datatype issue - Oracle Number(8) changed to INT when loaded to Kafka with Avro in Qlik Replicate

Datatype issue - Oracle Number(8) changed to INT when loaded to Kafka with Avro

Dane999 — Mon, 12 Sep 2022 17:36:26 GMT

I am using an Oracle source, loading to Kafka in Avro format. Use logical data types for specific data types is NOT checked.

This is causing my Number(8) column to get changed to an INT.

Are there any options i can use to avoid creating a transformation for each column that this applies to ?

Re: Datatype issue - Oracle Number(8) changed to INT when loaded to Kafka with Avro

SwathiPulagam — Mon, 12 Sep 2022 18:43:22 GMT

Hi @Dane999,

Below is the data type conversion from Oracle to Replicate:

From Replicate to Kafka:

If your precision is between 1 and 9 then Kafka Avro primitive datatype will be INT. So what are you expecting for the Target datatype?

And please provide the exact datatype on the source side.

Thanks,

Swathi

Re: Datatype issue - Oracle Number(8) changed to INT when loaded to Kafka with Avro

Dane999 — Mon, 12 Sep 2022 19:29:25 GMT

this is the Oracle source:

Kafka we are seeing:

Re: Datatype issue - Oracle Number(8) changed to INT when loaded to Kafka with Avro

shashi_holla — Tue, 27 Sep 2022 20:36:28 GMT

Hi @Dane999

Please provide more details on the actual issue with having INT on the Kafka side. Is it causing any data truncation?

Thank you,

Re: Datatype issue - Oracle Number(8) changed to INT when loaded to Kafka with Avro

SwathiPulagam — Thu, 29 Sep 2022 22:44:17 GMT

Hi @Dane999 ,

As I mentioned in my previous post it is working as described in the user guide.

What are you expecting in the Kafka target?

If you are expecting the same datatype like source as "number" then it is not a feasible option. Since Kafka has limited data types we will map with appropriate available datatypes in Kafka target.

Thanks,

Swathi