#### Archived

This topic is now archived and is closed to further replies.

# slight loss of precision

This topic is 5559 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

I''ve got a float that can only be between 0 and 1. I want this value as an integer so I can bit shift, so I''m multiplying by 1000 (cos the float is to 3 dec places), however doing this is causing a slight loss in precision. for example: 0.364 * 1000 = 363 When doing this however : float(0.364*1000) I get 363.999695 How can I fix this??

##### Share on other sites
your 0.364 is actually stored as 0.363999695

so if you cast 0.363999695 * 1000 to an int, you get 363

If you only need 3 digits after decimal point, you can do this:

float f = 0.364;
int i = int(f * 1000 + 0.5);

you can also use double s.

1. 1
2. 2
3. 3
4. 4
Rutin
12
5. 5

• 26
• 11
• 9
• 9
• 11
• ### Forum Statistics

• Total Topics
633700
• Total Posts
3013426
×

## Important Information

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!