Sign in to follow this  
ACCU

Weird problem [SOLVED] (I hate gcvt)

Recommended Posts

I use VC 6 float f; f=60.1; float a=f-(long)f; result: a is 0.0999985 when using 60.1-(long)60.1 result is 0.1 How you can explain that ? [Edited by - ACCU on July 10, 2005 4:49:14 AM]

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
rounding errors
and 60.1 is a double, 60.1f is a float.
fun fact: 0.1 cannot be exactly represented as a float/double as it is a not terminating fraction in base 2.

Share this post


Link to post
Share on other sites
that's incredible. just changed float to double and it worked. well... why for 60.1-60 flt precision is not enough that's the question. I wouldnt expect that

Share this post


Link to post
Share on other sites
Quote:
Original post by ACCU
that's incredible. just changed float to double and it worked. well... why for 60.1-60 flt precision is not enough that's the question. I wouldnt expect that


0.1 - 0.0999985 = 0.0000015

Not terribly much error there. Considering the original number is 60, that means the error is only about 0.0000025%, relative.

The problem is that numbers like 60.1, while nice in a base 10 system, dosn't directly translate into base 2. It's like trying to store the exact value of 1/3rd in base 10 decimal - you can approximate with 0.33333333, but that's still not exactly 1/3rd. In a base 3 system (digits 0,1, and 2) it's easy to store (1/3 (base-10 fractional) = 0.1 (base-3 decimal)).

You still have an error with a double, it's just that your output is truncating the result. Depending on what you're using to display your number, you may be able to manually set this. If you want this behavior with C++ iostreams using a float, just reduce the digits displayed. For example:
std::cout.setprecision( 3 );
float nearly_zero = 0.00001;
float nearly_one_nineth = 0.11111;
std::cout << nearly_zero << std::endl; //should print "0"
std::cout << nearly_one_nineth << std::endl; //should print "0.111"

Share this post


Link to post
Share on other sites
I've tried using sprintf(buf,"%.3f",f); and it works much better than gcvt.

now it is not making .09995 instead of .1
I see that because i edit my variable and show name with value in another place as well. Such behaviour was unacceptable. Now it's working correctly.
BTW, I even wrote my own conversion routine and it still worked similar .. :D Just couldn't leave like that.


Thank you all for explanation, especially MaulingMonkey. my respect.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this