Sign in to follow this  

accuracy error :s

This topic is 4597 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

ok when I divide 378 by 100 I get 7799999999999998 not 3.78 when I add 0.01 to 3.77 I get 7799999999999998 again. I need to get it as 3.78. my code is as simple as int a = 378; int b = pow(10.0f,2.0f); float c = a / b; this return c as 7799999999999998 or float a = 3.77; float b = 0.01; float c = a + b; this also returns 7799999999999998 any Idea as of what I can do. I'm using c++ with vs.net 2003

Share this post


Link to post
Share on other sites
Quote:
Original post by crazy_andy

float a = 3.77;
float b = 0.01;
float c = a + b;

this also returns 7799999999999998



How are you getting your return values- eg: how are you printing them out?

Share this post


Link to post
Share on other sites
It's sort of impossible to do what you want. You should go read up on floating point numbers, they rarely work out to be perfectly rounded. Whenever you're working with floats you want to use comparative operators not equality operators. Here is some information. Use google if that is insufficient.

Share this post


Link to post
Share on other sites
ok,thanks for your rapid replies.

scoota: I was viewing them while debugging
bincho: I want to display the inforrmation to the user, rather than compare it. (intresting read as well)
nhatkthanh: after a bit more messing, I displayed the result in a msg box and it comes out as 3.78000

so thats pretty good, except I need it to be just 3.78, anyone know how to do that?

thanks again

Share this post


Link to post
Share on other sites
Quote:
Original post by nhatkthanh
are you sure you're printing the right variable? I get the correct result, I don't see how it would give you that result due to precision.


I'm assuming that he's dropping the '3.' from what he "needs to get." Not sure which way is up, though.

Share this post


Link to post
Share on other sites
Quote:
Original post by crazy_andy
float b = 0.01;

It is impossible to store 1/100 in a float or double variable. Use decimal fixed point arithmetic instead.

For example, if you want to be able to store something like 1.46 dollars, simply store 146 cents. Got it?

Share this post


Link to post
Share on other sites
Quote:
Original post by crazy_andy
I want to display the inforrmation to the user [...]

so thats pretty good, except I need it to be just 3.78, anyone know how to do that?

Storing data and displaying data is two pairs of shoes. Just store 378 and display 3.78 like this:

int value = 378;
printf("%i.%02i", value/100, value%100);

Share this post


Link to post
Share on other sites
Quote:
Original post by Fred304
Quote:
Original post by crazy_andy
I want to display the inforrmation to the user [...]

so thats pretty good, except I need it to be just 3.78, anyone know how to do that?

Storing data and displaying data is two pairs of shoes. Just store 378 and display 3.78 like this:

int value = 378;
printf("%i.%02i", value/100, value%100);


...or simply:


float value=3.78;
printf("%0.2f", value);

Share this post


Link to post
Share on other sites
Quote:
Original post by ScootA
...or simply:


float value=3.78;
printf("%0.2f", value);

Yeah, but as I said, value will not exactly be 3.78 but slightly off. float is evil, you know? ;)

Share this post


Link to post
Share on other sites

This topic is 4597 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this