# Weird problem [SOLVED] (I hate gcvt)

This topic is 4825 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

I use VC 6 float f; f=60.1; float a=f-(long)f; result: a is 0.0999985 when using 60.1-(long)60.1 result is 0.1 How you can explain that ? [Edited by - ACCU on July 10, 2005 4:49:14 AM]

##### Share on other sites
60.1 is of type double, so you get more precision.

##### Share on other sites
rounding errors
and 60.1 is a double, 60.1f is a float.
fun fact: 0.1 cannot be exactly represented as a float/double as it is a not terminating fraction in base 2.

##### Share on other sites
that's incredible. just changed float to double and it worked. well... why for 60.1-60 flt precision is not enough that's the question. I wouldnt expect that

##### Share on other sites
Quote:
 Original post by ACCUthat's incredible. just changed float to double and it worked. well... why for 60.1-60 flt precision is not enough that's the question. I wouldnt expect that

0.1 - 0.0999985 = 0.0000015

Not terribly much error there. Considering the original number is 60, that means the error is only about 0.0000025%, relative.

The problem is that numbers like 60.1, while nice in a base 10 system, dosn't directly translate into base 2. It's like trying to store the exact value of 1/3rd in base 10 decimal - you can approximate with 0.33333333, but that's still not exactly 1/3rd. In a base 3 system (digits 0,1, and 2) it's easy to store (1/3 (base-10 fractional) = 0.1 (base-3 decimal)).

You still have an error with a double, it's just that your output is truncating the result. Depending on what you're using to display your number, you may be able to manually set this. If you want this behavior with C++ iostreams using a float, just reduce the digits displayed. For example:
std::cout.setprecision( 3 );float nearly_zero = 0.00001;float nearly_one_nineth = 0.11111;std::cout << nearly_zero << std::endl; //should print "0"std::cout << nearly_one_nineth << std::endl; //should print "0.111"

##### Share on other sites
I've tried using sprintf(buf,"%.3f",f); and it works much better than gcvt.

now it is not making .09995 instead of .1
I see that because i edit my variable and show name with value in another place as well. Such behaviour was unacceptable. Now it's working correctly.
BTW, I even wrote my own conversion routine and it still worked similar .. :D Just couldn't leave like that.

Thank you all for explanation, especially MaulingMonkey. my respect.

1. 1
2. 2
3. 3
Rutin
16
4. 4
JoeJ
13
5. 5

• 10
• 9
• 14
• 10
• 25
• ### Forum Statistics

• Total Topics
632646
• Total Posts
3007637
• ### Who's Online (See full list)

There are no registered users currently online

×