Sign in to follow this  

float precision difference between HLSL and GLSL ?!

Recommended Posts

I am trying to port a GLSL shader to HLSL, i finally managed to make it work, but I had to do some kind of hack i really don't like, and i guess I am probably missing something here (I'm new to this rendering world :-)).
So here is the thing, somewhere in the GLSL shader there is this line :
vec3 lambda = vec3(680E-9, 550E-9, 450E-9);
pow(lambda, vec3(4.0))
which i translated in HLSL as :
float3 lambda = float3(680E-9, 550E-9, 450E-9); ... pow(lambda, float3(4.0f,4.0f,4.0f))
I don't know why, but this gives different results in GLSL and HLSL. I know this, because when i turn the result of the pow() operator as a color, i get different colors in GLSL and HLSL :
// Fragment shader
void main() 

    float3 lambda = float3(680E-9, 550E-9, 450E-9);

    gl_FragColor.rgb = normalize(pow(lambda, vec3(4.0)));

    gl_FragColor.a = 1.0;


This returns a pure white color on whatever model i apply this shader on.
Now in HLSL :

float4 std_PS(vertexOutput IN) : SV_Target 

   float3 lambda = float3(680E-9, 550E-9, 450E-9);

   return float4(normalize(pow(lambda, float3(4.0f,4.0f,4.0f))), 1.0f);
This returns a light brown color on whatever model i apply this shader on.
I manually did the pow() using windows calculator, and i got this in my HLSL now:
float4 std_PS(vertexOutput IN) : SV_Target  
float3 powLambda = float3(0.00000000000000000000000021381376f, 0.00000000000000000000000009150625f, 0.00000000000000000000000004100625f);
return float4(normalize(powLambda), 1.0f);
And now i get the same white color i was getting in the GLSL shader, and the HLSL shader gives me exactly the same result the GLSL shader gives.
So why is happening ?
My guess is that there probably is a floating precision difference between GLSL and HLSL, but if it's indeed the case, how can deal with this in an elegant way without computing pow manually with my calculator and hardcoding results in my shader ?
Thanks ! Edited by Yann ALET

Share this post

Link to post
Share on other sites

pow() in HLSL uses some kind of approximation, pow(1, 10000) doesn't give white color either.


Have you tried to multiply 2 times?

lambda = lambda * lambda; // ^2
lambda = lambda * lambda; // ^2^2 = ^4



Edited by Zaoshi Kaba

Share this post

Link to post
Share on other sites

It is likely related only to compiler optimizations.  The precision of the floating-point values will be the same in either language on the same hardware as long as you are using the same precision specifiers in both languages.


If the compiler evaluates all of these equations at compile-time it evaluates to the following:

(0.00000000000000000000000021381376 * 0.00000000000000000000000021381376 + 0.00000000000000000000000009150625 * 0.00000000000000000000000009150625 + 0.00000000000000000000000004100625 * 0.00000000000000000000000004100625) = 5.57712302934626e-50

sqrt( 5.57712302934626e-50 ) = 2.3615933242932111574503306008555e-25

1.0 / 2.3615933242932111574503306008555e-25 = 4234429313943308758996590.5192989


4234429313943308758996590.5192989 * 0.00000000000000000000000021381376 = 0.90537925306843927260199484611165

4234429313943308758996590.5192989 * 0.00000000000000000000000009150625 = 0.38747674740902489712793176120659

4234429313943308758996590.5192989 * 0.00000000000000000000000004100625 = 0.173638067054887804798603939982


This is a nasty brown color that you described.

But look at these numbers.

No GPU can do this kind of math with that kind of precision, and sqrt() is only approximated anyway.  If this is evaluated at run-time, overflow leading to 1.0, 1.0, 1.0 (white) is obvious.



You should find another way around using such small numbers.  Even if it works for you it may not work for someone else with another version of the HLSL or GLSL compiler.

You can’t use these kinds of numbers reliably.



L. Spiro

Edited by L. Spiro

Share this post

Link to post
Share on other sites
Thank you both for your help !
Just so you know, the shader I'm porting is for a sky with atmosphere light scatering, and this thing plays with extremely small values.

@Zaoshi Kaba : The lambda * lambda * lambda * lambda trick worked .. ish ^^ It indeed gives me a white color, but the final effect is still broken compared to the original one in GLSL, so there is something fishy with this approach too :-/

@L. Spiro : I guess you're right, i will try to work around this.

Thanks !

Share this post

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this