Solved: How to handle data type conversion - Qlik Community

jdhruva · ‎2022-09-09

Experts need your guidance here. We are migrating from Oracle to Snowflake using QLIK. Part of the replication process we noticed that columns with datatype Number(20,2) in Oracle is transformed into Number(22) in Snowflake using QLIK. Data after the decimal is truncated in snowflake after a full/CDC load. Can you please help us fix this issue? Is there is a setting we need to configure to handle this transformation?

SwathiPulagam · ‎2022-11-09

Below is the solution we provided in the case :

Please refer to the Replicate user guide:

"
https://help.qlik.com/en-US/replicate/May2022/Content/Replicate/Main/Oracle/ora_source_data_types.ht...

NUMBER (P,S)

NUMBER according to the "Expose number as" property in the Qlik Replicate Oracle source database settings. "

NUMBER in oracle to Replicate handled as described in the document. There is no internal NUMBER datatype in replicate so we can't handle dynamic changes in the datatype as like Oracle. To Overcome this issue, we introduced "Expose number as" property in Oracle endpoint.

Ex: If NUMBER is defined with specific Precision and Scale on the source then replicate will create same NUMBER(P, S) on target.
If no Precision and Scale is defined on the source then as we explained replicate can't decide what would be the length. You have to review all your NUMBER columns and decide the best value for the "Expose number as" property.
If a different NUMBER column has to be defined with different Precision and Scale then it has to be set at table level transformation.

Please create a feature request with your use case. The product manager will review and prioritize the request.

Thanks,

Swathi

View solution in original post

SwathiPulagam · ‎2022-09-09

Hi @jdhruva,

This is not the expected behavior. I just tested and it is transferring the same datatype as it is in Oracle.

Example: Oracle source datatype Number(38,0)

Snowflake also same Number(38,0)

Thanks,

Swathi

SwathiPulagam · ‎2022-09-09

Hi @jdhruva,

I would recommend you to create a support case for doing more analysis.

Thanks,

Swathi

shashi_holla · ‎2022-09-09

Hi @jdhruva

In addition to what @SwathiPulagam said, as per the documentation the behavior that you mentioned can happen only when the scale is between 38 and 127 else it will match the same data type as in the source Oracle. In your case is Number(20,2) should be the same in the target as well.

jdhruva · ‎2022-09-09

Hello Swathi, We I guess the team has already submitted a case. Here is another example, where the conversion didn't happen the right way (Refer Attached Screenshot; it is comparison between SNowflake and Oracle after a full load.Left column is Snowflake and Right column is Oracle)

SwathiPulagam · ‎2022-09-09

Hi @jdhruva ,

Can you share the case number?

Thanks,

Swathi

SwathiPulagam · ‎2022-09-09

Hi @jdhruva ,

Also please check if you enable any Global transformation to convert the datatype.

Thanks,

Swathi

jdhruva · ‎2022-09-09

Yes we have global transformation configured NUMERIC to 15,0

jdhruva · ‎2022-09-09

case# 00051525

jdhruva · ‎2022-09-09

Thanks Shashi. I assume this is already defined QLIK by default

How to handle data type conversion

Best Practices

Configuration