I have an issue with the floating point calculation as I suspect QV does something wired when a tried to subtract a 5 digit fraction value with another 5 digit fraction.
I tried and simplifed the problem to this:
When I create a text object with =70.1-70.0, I am getting 0.099999999999994 instead of .1. I tried this on QV 10 SR2.
I am not sure if any one faced this issue. Surely this can't be expected behaviour as a simple windows calculator gets me the correct value.
in binary floating point representation, you can't represent all values exactely, so for some values or some results of a calculation, you will get a very small calculation error.
I believe it's just up to some "intelligent rounding", if you'll notice this error or not.
For example
=num(70.1-70.0,'#')
will show 0.1, which seems to be correct, but I am pretty sure the internal number representation will still show the small error. And I also believe Windows calculator suffers from the same problem, it might just be a little more clever showing you rounded values.
