accuracy error :s
Author
ok when I divide 378 by 100 I get 7799999999999998 not 3.78
when I add 0.01 to 3.77 I get 7799999999999998 again.
I need to get it as 3.78.
my code is as simple as
int a = 378;
int b = pow(10.0f,2.0f);
float c = a / b;
this return c as 7799999999999998
or
float a = 3.77;
float b = 0.01;
float c = a + b;
this also returns 7799999999999998
any Idea as of what I can do. I'm using c++ with vs.net 2003
Quote:Original post by crazy_andy
float a = 3.77;
float b = 0.01;
float c = a + b;
this also returns 7799999999999998
How are you getting your return values- eg: how are you printing them out?
It's sort of impossible to do what you want. You should go read up on floating point numbers, they rarely work out to be perfectly rounded. Whenever you're working with floats you want to use comparative operators not equality operators. Here is some information. Use google if that is insufficient.
are you sure you're printing the right variable? I get the correct result, I don't see how it would give you that result due to precision.
I got the same kind of errors when I was using printf() wrong. How was I supposed to know that ints and int64 had different letters? ;)
Good luck.
Good luck.
Author
ok,thanks for your rapid replies.
scoota: I was viewing them while debugging
bincho: I want to display the inforrmation to the user, rather than compare it. (intresting read as well)
nhatkthanh: after a bit more messing, I displayed the result in a msg box and it comes out as 3.78000
so thats pretty good, except I need it to be just 3.78, anyone know how to do that?
thanks again
scoota: I was viewing them while debugging
bincho: I want to display the inforrmation to the user, rather than compare it. (intresting read as well)
nhatkthanh: after a bit more messing, I displayed the result in a msg box and it comes out as 3.78000
so thats pretty good, except I need it to be just 3.78, anyone know how to do that?
thanks again
Quote:Original post by nhatkthanh
are you sure you're printing the right variable? I get the correct result, I don't see how it would give you that result due to precision.
I'm assuming that he's dropping the '3.' from what he "needs to get." Not sure which way is up, though.
Quote:Original post by crazy_andy
float b = 0.01;
It is impossible to store 1/100 in a float or double variable. Use decimal fixed point arithmetic instead.
For example, if you want to be able to store something like 1.46 dollars, simply store 146 cents. Got it?
Quote:Original post by crazy_andy
I want to display the inforrmation to the user [...]
so thats pretty good, except I need it to be just 3.78, anyone know how to do that?
Storing data and displaying data is two pairs of shoes. Just store 378 and display 3.78 like this:
int value = 378;printf("%i.%02i", value/100, value%100);Quote:Original post by Fred304Quote:Original post by crazy_andy
I want to display the inforrmation to the user [...]
so thats pretty good, except I need it to be just 3.78, anyone know how to do that?
Storing data and displaying data is two pairs of shoes. Just store 378 and display 3.78 like this:int value = 378;printf("%i.%02i", value/100, value%100);
...or simply:
float value=3.78;printf("%0.2f", value);This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
