Qlik Community

QlikView App Development

Discussion Board for collaboration related to QlikView App Development.

sundarakumar
Valued Contributor II

Calculation logic in text box


Hi Experts,

I tried writing the below in a text box

=101.02-101.01

The result 0.0099999999999909.

What could be the reason for this, how is this calculation being made in the back end.

Am using SR12.

Thanks in advance.

-Sundar

1 Solution

Accepted Solutions
Highlighted
MVP
MVP

Re: Calculation logic in text box

QlikView is using 8 byte IEEE 754 double precison floating point number format. It does not represent every possible value accurately. So the number you get is correctly calculated according to the number format and precision you can expect. You should use rounding for display and final reporting purposes. Have a look at Wikipedia and other sources about IEEE 754. Microsoft Excel also stick to this standard as most commercial software also does. It is a very good trade-off between speed, space and necessary precision.

View solution in original post

6 Replies
Highlighted
Employee
Employee

Re: Calculation logic in text box

Use round

=round(101.02-101.01,0.01)

Highlighted
MVP
MVP

Re: Calculation logic in text box

What could be the reason for this, how is this calculation being made in the back end.

Rounding Errors

Highlighted
MVP
MVP

Re: Calculation logic in text box

QlikView is using 8 byte IEEE 754 double precison floating point number format. It does not represent every possible value accurately. So the number you get is correctly calculated according to the number format and precision you can expect. You should use rounding for display and final reporting purposes. Have a look at Wikipedia and other sources about IEEE 754. Microsoft Excel also stick to this standard as most commercial software also does. It is a very good trade-off between speed, space and necessary precision.

View solution in original post

Highlighted
MVP
MVP

Re: Calculation logic in text box

If you try the exact same calculation in Excel and increase the number of displayed decimals sufficiently you will get the same number in fact:

0,0099999999999909
Highlighted
sundarakumar
Valued Contributor II

Re: Calculation logic in text box

Thanks All lot more to learn

Highlighted
sundarakumar
Valued Contributor II

Re: Calculation logic in text box

Thanks for the reply, wanted to know what the logic was. Got it