# A peculiar rounding error

This topic is 4459 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

Windows is struggling to be precise (or more specifically, the command shell in Windows XP) to 2 d.p. Here's a test code that multiplies 1105 by 5.7 (then divides by 100.0) and sets the precision to 2 d.p. However, instead of yielding 62.985 as the answer, it's storing as 62.98499... and rounding to 62.98 to 2 d.p I've tried this on two PCs so far - any comments and help appreciated (I need better precision to 2 d.p as the unit digit represents pence in financial terms)

#include <iostream>
#include <iomanip>

using namespace std;

int main()
{
int x;
double y;

x = 1105;
y = 5.700;
// Any calculator worth it's salt (or any paper/pencil arithmetic) shows
// that x*y/100 = 62.985 exactly. To 2 d.p it is 62.99

cout << setiosflags(std::ios::fixed | std::ios::showpoint)
<< setprecision(2) << x*y/100.0 << endl;

// This rounds it to 62.98 (fixed format, to 2 d.p)
// It's worth noting that to 3 d.p it correctly gives 62.985

return 0;
}



##### Share on other sites
try casting x to float or double before you do anything with it. The left hand side type is pretty significant for this kind of thing, and x is just an int
e.g.

<< setprecision(2) << ((double)x)*y/100.0 << endl;

##### Share on other sites
You will always get rounding precision errors because of the IEEE float memory respresentation..
The best you can do is to use higher precision data type, double etc....

##### Share on other sites
Yeh - I've tried all kinds of casting (that included) - even long doubles, but it still gives 62.98 as the answer. It seems to have a low precision of about 5 sig. figures.

Awful.

##### Share on other sites
That (paulecoyote's reply) won't help. x will automatically be converted to double since it is multiplied by a double. The problem is that floating point numbers cannot store most numbers exactly. The sacrifice precision for much greater range. Instead of using a floating point variable for what is actually a discrete quantity just store the number of pence as an integral value. Then to print it you can just do:
cout << (pence / 100) << '.' << width(2) << setfill('0') << (pence % 100) << endl;
EDIT: actually, rereading your post, I'm not sure if this is sufficient. I had assumed that x was a quantity and y was a price in pounds, but it clearly isn't (since you divide by 100). However the principle of fixed-point arithmetic may still be applicable.

Enigma

##### Share on other sites
Or, you could just add 0.005f to your result, before displaying it, then it would round correctly

##### Share on other sites
Quote:
 I had assumed that x was a quantity and y was a price in pounds, but it clearly isn't (since you divide by 100).

The 5.7 value is the cost in pence whilst the 1105 is the number of units (it's actually part of an energy bill calculation)- so the final value will have a fractional part of pence.

##### Share on other sites
Quote:
 Or, you could just add 0.005f to your result, before displaying it, then it would round correctly

I've just tried that and it works on 62.985. However, it's rounding up another number so that it's 1 pence too high :/

##### Share on other sites
Why is because floating point numbers are stored in base 2. Just like why you can't write 1/3 out exacly in base 10. So if you had to use limited percision decimal math floor(1/3+1/3+1/3)==0. There's "more" numbers that have this problem in base 2 then base 10 (because 5 divides 10 but not 2). For instance .1.

As for fixing this it kinda depends on what your trying to solve. Doing all the math with ints and then adding the decimal when you write could work. Or using rational numbers. If you start adding in irrationals (pi or e), then those wont work. Windows calc uses something they call extend precision (from the calc help it looks like it uses ratinals until you use an irrational number.

##### Share on other sites
Either use a higher precision floating point type or switch to a decimal representation.

##### Share on other sites
One potential solution is to use integers instead, in a kind of base-10 fixed point:
typedef unsigned int Fixed;const unsigned int FixedDivisor = 1000;int FixedWholePart(Fixed Number){  return Number / FixedDivisor;}//Only sow 2 digits normally, the last one is only for roundingint FixedFractionPart(Fixed Number, int Digits = 2){  int Fraction = Number % FixedDivisor;  while(Digits < 3)  {    Fraction = (Fraction + 5) / 10;  }  return Fraction;}Fixed PriceInPence = 5700;int Quantity = 1105;Fixed PriceInPounds = (Quantity * PriceInPence) / 100;cout << FixedWholePart(PriceInPounds) << "." << FixedFractionPart() << endl;
Of course, a real implementation would probably be better as a class with all the operators overloaded. For multiplication, you need to do (A * B) / FixedDivisor, or something with the same effect (like divide A and B by the square root of FixedDivisor) because otherwise the divisor will be squared and your number will come out far too large. The opposite must be done for division - usually (A * FixedDivisor) / B - so the divisor isn't eliminated.

-Extrarius

##### Share on other sites
Here's the "I've been reading through the boost docs and I've been looking for uses for some of that stuff" version.

#include <iostream>#include <iomanip>#include <boost/numeric/interval.hpp>using namespace std;using namespace boost::numeric;using namespace interval_lib;typedef interval<double> Inter;int main(){    Inter x(1105);    Inter y(5.7);    cout << setiosflags(std::ios::fixed | std::ios::showpoint)      << setprecision(2) << upper(x*y/100.0) << endl;}

Note the interval stuff modifies the FPU's rounding settings. And comparisons work in odd ways. And there's a couple other quirks. Possible speed issues. To be honest this is the first thing I ever used them in.

And I don't know if intervals are the best solution to this, I just like them.

[Edited by - Cocalus on November 4, 2005 10:37:38 PM]

##### Share on other sites
Using any floating point representation for financial data is a bad idea.

Much better to use a pure decimal / BCD representation.

##### Share on other sites
Quote:
 Original post by Anonymous PosterUsing any floating point representation for financial data is a bad idea.Much better to use a pure decimal / BCD representation.
That's right. This is precisely why VB has a currency data type.

This works just fine, putting in a round() call:
double round(double a){    return (a > 0) ? floor(a+0.5) : ceil(a-0.5);}...    cout << setiosflags(std::ios::fixed | std::ios::showpoint)      << setprecision(2) << round(x*y)/100.0 << endl;
and if you think about it, it's exactly what you want to do.