Jump to content
  • Advertisement
Sign in to follow this  
griminventions

Floating point rounding woes

This topic is 5461 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Is there a good way to chop off the value of a float beyond a certain accuracy? I want to get rid of anything smaller than 0.001 but keep the rest of the value. For example, 2.1005 would become 2.100. (I'm doing this in C++, by the way.)

Share this post


Link to post
Share on other sites
Advertisement
For a given precision p,
Find the magnitude of your number m=⌊log10 |x|⌋
Divide the number by 10m-p
Round off
Multiply by 10m-p

Would that do?

Share this post


Link to post
Share on other sites
Quote:
Original post by griminventions
Yowza, that looks computationally expensive, but if there's no other way. Thanks for the reply!


There can very well be, that's just the first thing that came to my mind.

Share this post


Link to post
Share on other sites
I came to the conclusion that instead of chopping out the tiny fractions giving me problems, I can ignore values below a threshold to solve the problem effectively. But thanks for the replies!

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
It's not very computation expensive. However, a float cannot represent a value like 2.1 exactly, so you will still only get an approximation of that value.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
floorf (x * 1000.0f) * 0.0001f;

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Mi mistake
floorf (x * 1000.0f) * 0.001f;

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!