Floating point rounding woes

Math and Physics Programming

Started by griminventions October 04, 2004 05:23 PM

9 comments, last by Charles B 19 years, 6 months ago

170

Author

October 04, 2004 05:23 PM

Is there a good way to chop off the value of a float beyond a certain accuracy? I want to get rid of anything smaller than 0.001 but keep the rest of the value. For example, 2.1005 would become 2.100. (I'm doing this in C++, by the way.)

--JasonGrim Inventions Interactive

Fruny

1,658

October 04, 2004 05:29 PM

For a given precision p,
Find the magnitude of your number m=⌊log10 |x|⌋
Divide the number by 10m-p
Round off
Multiply by 10m-p

Would that do?

"Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." — Brian W. Kernighan

griminventions

170

Author

October 04, 2004 05:36 PM

Yowza, that looks computationally expensive, but if there's no other way. Thanks for the reply!

--JasonGrim Inventions Interactive

Fruny

1,658

October 04, 2004 05:42 PM

Quote:Original post by griminventions
Yowza, that looks computationally expensive, but if there's no other way. Thanks for the reply!

There can very well be, that's just the first thing that came to my mind.

griminventions

170

Author

October 04, 2004 05:51 PM

I came to the conclusion that instead of chopping out the tiny fractions giving me problems, I can ignore values below a threshold to solve the problem effectively. But thanks for the replies!

--JasonGrim Inventions Interactive

Anonymous

October 04, 2004 05:55 PM

It's not very computation expensive. However, a float cannot represent a value like 2.1 exactly, so you will still only get an approximation of that value.