Sign in to follow this  
RenHoek

Floating Point Precision in pixel shader

Recommended Posts

Hi guys

So I'm working on some stuff atm which requires random numbers within a pixel shader.
I'm aiming for DX9 hardware so integer hashing is not an option.
I've opted for a floating point hash from the x,y integer pixel coordinate of the form

hash = mod( x*x*y*y, SOMEPRIME ) / SOMEPRIME

I seem to be getting good results from it. ( I can clamp and offset the domain to avoid the reflection along the x=y diagonal line )

Thing is, the x*x*y*y operation pushes the floating point integer coordinate way past ( 2 << 23 ). So the floating point number starts to loose precision.
eg, x = 99.0 y = 99.0 99.0^4 > 2.0^23

So my questions.
Do modern graphics cards conform to the IEEE 32bit floating point standard?
Are there any that do not? ( I think I remember some ATI cards being 24 bit at some point... )
Can I expect this function to produce the same results across different cards? At least the ones which are IEEE 32bit FP compliant right?

thoughts?

Thanks all!
Ren

Share this post


Link to post
Share on other sites
Any SM3.0-capable GPU will use 32-bit floating point precision by default. The old Nvidia SM3.0 GPU's (6000-series and 7000-series) could also run half-precision instructions, but it won't use them unless you use half primitives in your shader. By default they won't fully conform to IEEE floating point rules, but you can force this if you wish through compiler flags.

ATI SM2.0 GPU's (9000 series and X series) used 24-bit precision since that was the minimum required by that shading model, so no guarantees on that hardware.

Share this post


Link to post
Share on other sites
What if you split it up like:

part1 = [color=#1C2837][size=2]mod( x*x, [/size][/color][color=#1C2837][size=2]PRIME1 [/size][/color][color=#1C2837][size=2]);[/size][/color]
[color=#1C2837][size=2]part2 = [/size][/color][color=#1C2837][size=2]mod( y*y, PRIME2[/size][/color][color=#1C2837][size=2] );[/size][/color]
[color=#1C2837][size=2]hash = mod( part1 * part2[/size][/color][color=#1C2837][size=2], PRIME3[/size][/color][color=#1C2837][size=2])[/size][/color][color=#1C2837][size=2] / PRIME3[/size][/color][color=#1C2837][size=2];[/size][/color]

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this