Jump to content
  • Advertisement
Sign in to follow this  

Floating Point Precision in pixel shader

This topic is 2927 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi guys

So I'm working on some stuff atm which requires random numbers within a pixel shader.
I'm aiming for DX9 hardware so integer hashing is not an option.
I've opted for a floating point hash from the x,y integer pixel coordinate of the form

hash = mod( x*x*y*y, SOMEPRIME ) / SOMEPRIME

I seem to be getting good results from it. ( I can clamp and offset the domain to avoid the reflection along the x=y diagonal line )

Thing is, the x*x*y*y operation pushes the floating point integer coordinate way past ( 2 << 23 ). So the floating point number starts to loose precision.
eg, x = 99.0 y = 99.0 99.0^4 > 2.0^23

So my questions.
Do modern graphics cards conform to the IEEE 32bit floating point standard?
Are there any that do not? ( I think I remember some ATI cards being 24 bit at some point... )
Can I expect this function to produce the same results across different cards? At least the ones which are IEEE 32bit FP compliant right?


Thanks all!

Share this post

Link to post
Share on other sites
Any SM3.0-capable GPU will use 32-bit floating point precision by default. The old Nvidia SM3.0 GPU's (6000-series and 7000-series) could also run half-precision instructions, but it won't use them unless you use half primitives in your shader. By default they won't fully conform to IEEE floating point rules, but you can force this if you wish through compiler flags.

ATI SM2.0 GPU's (9000 series and X series) used 24-bit precision since that was the minimum required by that shading model, so no guarantees on that hardware.

Share this post

Link to post
Share on other sites
What if you split it up like:

part1 = [color=#1C2837][size=2]mod( x*x, [color=#1C2837][size=2]PRIME1 [color=#1C2837][size=2]);
[color=#1C2837][size=2]part2 = [color=#1C2837][size=2]mod( y*y, PRIME2[color=#1C2837][size=2] );
[color=#1C2837][size=2]hash = mod( part1 * part2[color=#1C2837][size=2], PRIME3[color=#1C2837][size=2])[color=#1C2837][size=2] / PRIME3[color=#1C2837][size=2];

Share this post

Link to post
Share on other sites
Sign in to follow this  

  • Advertisement

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!