Sign in to follow this  

float error problem shader 4

This topic is 1915 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

does the float can handle 1/8000 numbers and multiples of it? from 1/8000 to 8000/8000 and then plus 1.0f or -1.0f

and then adding some multiples together, the anwer is another multiple but can still handle it?

rendering to a 2D texture that is 8000x8000, I think a bug of my app is because of float error.

I just relized its not 1/8000 but 1/16000 what iam using when the texture that is 8000x8000 Edited by lomateron

Share this post


Link to post
Share on other sites
If I remember correctly float has 24 bits of precision, so it should have no problems representing 1 / 16,777,216.

However, it cannot represent all the number inbetween.
1/2 - can
1/3 - cannot
1/4 - can
1/5 - cannot
1/6 - cannot
1/7 - cannot
1/8 - can
and so on...

If you add 1.0f or -1.0f to these numbers it'll only change few last digits as decimal accuracy will decrease. It'll be around
1.cccccccww
where c - correct digit, w - incorrect.

Share this post


Link to post
Share on other sites
Sign in to follow this