1 reply to this topic

[lomateron]

does the float can handle 1/8000 numbers and multiples of it? from 1/8000 to 8000/8000 and then plus 1.0f or -1.0f

and then adding some multiples together, the anwer is another multiple but can still handle it?

rendering to a 2D texture that is 8000x8000, I think a bug of my app is because of float error.

I just relized its not 1/8000 but 1/16000 what iam using when the texture that is 8000x8000

Edited by lomateron, 19 October 2012 - 01:35 PM.

[Ripiz]

If I remember correctly float has 24 bits of precision, so it should have no problems representing 1 / 16,777,216.

However, it cannot represent all the number inbetween.
1/2 - can
1/3 - cannot
1/4 - can
1/5 - cannot
1/6 - cannot
1/7 - cannot
1/8 - can
and so on...

If you add 1.0f or -1.0f to these numbers it'll only change few last digits as decimal accuracy will decrease. It'll be around
1.cccccccww
where c - correct digit, w - incorrect.