2 Replies Latest reply: May 31, 2017 12:05 PM by Todd Buss

# conceptual issue with decimal rounding errors

I'm new to Qlik.  Pardon my lack of understanding of data processing.  I have a computation method that is amplifying a rounding error to the point of making the results unusable.  Let me try to describe my data in simplified form:

Every week, hospital sites make referrals.  Each week the hospital populations can change.  I tied a population database value to each of the referrals.  The result is a table similar to this:

WeekPatientPopulation of patient's hospital during the week (from a population table)

Fraction of Population (calculated)

Decimal of Population (Calculated)
1Jones3001/3000.003333
1Smith3001/3000.003333
2Thomas3501/3500.002857
3Smith4001/4000.0025
3Harding4001/4000.0025
3Trump4001/400

0.0025

In theory, if I add up all the decimal calculations (5th column), I should get a value that represents the referral rate for the hospital with some accounting for the fluctuation in population.  Unfortunately this value is different from what I get when I add the equivalent decimals from the 4th column.  When dealing with a thousand patients, and computing referrals per population per year at each facility, the difference between fractions and decimals becomes apparent.

I suspect I'll need to fix this by dynamically computing the average population for the hospital, based on the weeks selected by the Qlik user, and then use this as the denominator in computing a rate.  It just seems like my original method outlined above should be more accurate.  I'm open to alternative approaches.

I hope this is clear. Thanks in advance to anyone considering this.

--Todd

• ###### Re: conceptual issue with decimal rounding errors

Take a look here: Rounding Errors.

- Marcus

• ###### Re: conceptual issue with decimal rounding errors

Thanks.  Your reply helps to outline the problem.