topic Re: num() rounds differently to round() for half-step values in Archived Groups

num() rounds differently to round() for half-step values

gussfish — Wed, 23 Jul 2025 14:48:52 GMT

Hi folks,

I've been doing some work on writing rounded numbers to file (for which I've used num(), and then providing a reconciliation to itemise the resulting rounding error (for which I've used round()). But I've discovered that num() rounds differently to round(), which ruins my reconciliation! Obviously, I already know how to workaround this problem to ensure my reconciliation works; but of course I'm interested in this from the bug perspective to see if you agree that this is a bonfide bug that I should report, and whether it's a batle worth fighting for the Community.

This bug concerns values that are half-steps i.e. exactly half-way between two consecutive rounded values e.g. when rounding to 1 decimal place, how should 3.5 be rounded? Round() is consistent in rounding such numbers towards zero i.e. positive half-step numbers get rounded down and negative half-step numbers get rounded up. Sometimes it goes the other way; but I accept that this is likely due to the numbers have a tiny amount of machine fp error that sends the rounding the other direction.

Num(), though, from the small amount of testing I've down, consistently rounds negative half-steps down, while positive ones have no obvious pattern as to which ones round up and which ones round down.

See attached QVW for tabulation of these different behaviours.

Your thoughts?

Angus.

Re: num() rounds differently to round() for half-step values

rbecher — Tue, 19 Nov 2013 06:51:39 GMT

Hi Angus,

I also discovered similar problems with round() on half-step values: Rounding Issues

So, round() doesn't work consistent either:

round(2.05, 0.1) => 2.0 should be 2.1

round(2.15, 0.1) => 2.1 should be 2.2

round(2.25, 0.1) => 2.3 ok

round(2.35, 0.1) => 2.3 should be 2.4

round(2.45, 0.1) => 2.4 should be 2.5

To solve this (floating-point arithmetic issue) I used the round on an integer value:

round(2.15 * 10, 1) / 10 => 2.2 ok

But, I think this should be fixed inside the round() function!

- Ralf

Re: Re: num() rounds differently to round() for half-step values

rbecher — Tue, 19 Nov 2013 07:02:49 GMT

To illustrate I've added Integer Rounding to your app.

Re: num() rounds differently to round() for half-step values

rbecher — Tue, 19 Nov 2013 07:10:03 GMT

However, results seems to be also version related. And now I get even different results than as I've posted the original post a few month ago..