Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Join us in Toronto Sept 9th for Qlik's AI Reality Tour! Register Now
cancel
Showing results for 
Search instead for 
Did you mean: 
gussfish
Creator II
Creator II

num() rounds differently to round() for half-step values


Hi folks,

I've been doing some work on writing rounded numbers to file (for which I've used num(), and then providing a reconciliation to itemise the resulting rounding error (for which I've used round()).  But I've discovered that num() rounds differently to round(), which ruins my reconciliation!  Obviously, I already know how to workaround this problem to ensure my reconciliation works; but of course I'm interested in this from the bug perspective to see if you agree that this is a bonfide bug that I should report, and whether it's a batle worth fighting for the Community.

This bug concerns values that are half-steps i.e. exactly half-way between two consecutive rounded values e.g. when rounding to 1 decimal place, how should 3.5 be rounded?  Round() is consistent in rounding such numbers towards zero i.e. positive half-step numbers get rounded down and negative half-step numbers get rounded up.  Sometimes it goes the other way; but I accept that this is likely due to the numbers have a tiny amount of machine fp error that sends the rounding the other direction.

Num(), though, from the small amount of testing I've down, consistently rounds negative half-steps down, while positive ones have no obvious pattern as to which ones round up and which ones round down.

See attached QVW for tabulation of these different behaviours.

Your thoughts?

Angus.

Labels (2)
3 Replies
rbecher
MVP
MVP

Hi Angus,

I also discovered similar problems with round() on half-step values: Rounding Issues

So, round() doesn't work consistent either:

round(2.05, 0.1) => 2.0 should be 2.1

round(2.15, 0.1) => 2.1 should be 2.2

round(2.25, 0.1) => 2.3 ok

round(2.35, 0.1) => 2.3 should be 2.4

round(2.45, 0.1) => 2.4 should be 2.5

To solve this (floating-point arithmetic issue) I used the round on an integer value:

round(2.15 * 10, 1) / 10  => 2.2 ok


But, I think this should be fixed inside the round() function!

- Ralf

Astrato.io Head of R&D
rbecher
MVP
MVP

To illustrate I've added Integer Rounding to your app.

Astrato.io Head of R&D
rbecher
MVP
MVP

However, results seems to be also version related. And now I get even different results than as I've posted the original post a few month ago..

Astrato.io Head of R&D