Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi folks,
I've been doing some work on writing rounded numbers to file (for which I've used num(), and then providing a reconciliation to itemise the resulting rounding error (for which I've used round()). But I've discovered that num() rounds differently to round(), which ruins my reconciliation! Obviously, I already know how to workaround this problem to ensure my reconciliation works; but of course I'm interested in this from the bug perspective to see if you agree that this is a bonfide bug that I should report, and whether it's a batle worth fighting for the Community.
This bug concerns values that are half-steps i.e. exactly half-way between two consecutive rounded values e.g. when rounding to 1 decimal place, how should 3.5 be rounded? Round() is consistent in rounding such numbers towards zero i.e. positive half-step numbers get rounded down and negative half-step numbers get rounded up. Sometimes it goes the other way; but I accept that this is likely due to the numbers have a tiny amount of machine fp error that sends the rounding the other direction.
Num(), though, from the small amount of testing I've down, consistently rounds negative half-steps down, while positive ones have no obvious pattern as to which ones round up and which ones round down.
See attached QVW for tabulation of these different behaviours.
Your thoughts?
Angus.
Hi Angus,
I also discovered similar problems with round() on half-step values: Rounding Issues
So, round() doesn't work consistent either:
round(2.05, 0.1) => 2.0 should be 2.1
round(2.15, 0.1) => 2.1 should be 2.2
round(2.25, 0.1) => 2.3 ok
round(2.35, 0.1) => 2.3 should be 2.4
round(2.45, 0.1) => 2.4 should be 2.5
To solve this (floating-point arithmetic issue) I used the round on an integer value:
round(2.15 * 10, 1) / 10 => 2.2 ok
But, I think this should be fixed inside the round() function!
- Ralf
To illustrate I've added Integer Rounding to your app.
However, results seems to be also version related. And now I get even different results than as I've posted the original post a few month ago..