does the float can handle 1/8000 numbers and multiples of it? from 1/8000 to 8000/8000 and then plus 1.0f or -1.0f
and then adding some multiples together, the anwer is another multiple but can still handle it?
rendering to a 2D texture that is 8000x8000, I think a bug of my app is because of float error.
I just relized its not 1/8000 but 1/16000 what iam using when the texture that is 8000x8000
float error problem shader 4
If I remember correctly float has 24 bits of precision, so it should have no problems representing 1 / 16,777,216.
However, it cannot represent all the number inbetween.
1/2 - can
1/3 - cannot
1/4 - can
1/5 - cannot
1/6 - cannot
1/7 - cannot
1/8 - can
and so on...
If you add 1.0f or -1.0f to these numbers it'll only change few last digits as decimal accuracy will decrease. It'll be around
1.cccccccww
where c - correct digit, w - incorrect.
However, it cannot represent all the number inbetween.
1/2 - can
1/3 - cannot
1/4 - can
1/5 - cannot
1/6 - cannot
1/7 - cannot
1/8 - can
and so on...
If you add 1.0f or -1.0f to these numbers it'll only change few last digits as decimal accuracy will decrease. It'll be around
1.cccccccww
where c - correct digit, w - incorrect.
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement