Jump to content

  • Log In with Google      Sign In   
  • Create Account


We have 4 x Pro Licences (valued at $59 each) for 2d modular animation software Spriter to give away in this Thursday's GDNet Direct email newsletter.

Read more in this forum topic or make sure you're signed up (from the right-hand sidebar on the homepage) and read Thursday's newsletter to get in the running!

float precision difference between HLSL and GLSL ?!

Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
3 replies to this topic

#1 Yann ALET   Members   -  Reputation: 224


Posted 26 December 2012 - 11:12 PM

I am trying to port a GLSL shader to HLSL, i finally managed to make it work, but I had to do some kind of hack i really don't like, and i guess I am probably missing something here (I'm new to this rendering world :-)).
So here is the thing, somewhere in the GLSL shader there is this line :
vec3 lambda = vec3(680E-9, 550E-9, 450E-9);
pow(lambda, vec3(4.0))
which i translated in HLSL as :
float3 lambda = float3(680E-9, 550E-9, 450E-9); ... pow(lambda, float3(4.0f,4.0f,4.0f))
I don't know why, but this gives different results in GLSL and HLSL. I know this, because when i turn the result of the pow() operator as a color, i get different colors in GLSL and HLSL :
// Fragment shader
void main() 

    float3 lambda = float3(680E-9, 550E-9, 450E-9);

    gl_FragColor.rgb = normalize(pow(lambda, vec3(4.0)));

    gl_FragColor.a = 1.0;


This returns a pure white color on whatever model i apply this shader on.
Now in HLSL :

float4 std_PS(vertexOutput IN) : SV_Target 

   float3 lambda = float3(680E-9, 550E-9, 450E-9);

   return float4(normalize(pow(lambda, float3(4.0f,4.0f,4.0f))), 1.0f);
This returns a light brown color on whatever model i apply this shader on.
I manually did the pow() using windows calculator, and i got this in my HLSL now:
float4 std_PS(vertexOutput IN) : SV_Target  
float3 powLambda = float3(0.00000000000000000000000021381376f, 0.00000000000000000000000009150625f, 0.00000000000000000000000004100625f);
return float4(normalize(powLambda), 1.0f);
And now i get the same white color i was getting in the GLSL shader, and the HLSL shader gives me exactly the same result the GLSL shader gives.
So why is happening ?
My guess is that there probably is a floating precision difference between GLSL and HLSL, but if it's indeed the case, how can deal with this in an elegant way without computing pow manually with my calculator and hardcoding results in my shader ?
Thanks !

Edited by Yann ALET, 26 December 2012 - 11:16 PM.


#2 Zaoshi Kaba   Crossbones+   -  Reputation: 4606


Posted 27 December 2012 - 01:28 AM

pow() in HLSL uses some kind of approximation, pow(1, 10000) doesn't give white color either.


Have you tried to multiply 2 times?

lambda = lambda * lambda; // ^2
lambda = lambda * lambda; // ^2^2 = ^4



Edited by Zaoshi Kaba, 27 December 2012 - 01:30 AM.

#3 L. Spiro   Crossbones+   -  Reputation: 14286


Posted 27 December 2012 - 01:31 AM

It is likely related only to compiler optimizations.  The precision of the floating-point values will be the same in either language on the same hardware as long as you are using the same precision specifiers in both languages.


If the compiler evaluates all of these equations at compile-time it evaluates to the following:

(0.00000000000000000000000021381376 * 0.00000000000000000000000021381376 + 0.00000000000000000000000009150625 * 0.00000000000000000000000009150625 + 0.00000000000000000000000004100625 * 0.00000000000000000000000004100625) = 5.57712302934626e-50

sqrt( 5.57712302934626e-50 ) = 2.3615933242932111574503306008555e-25

1.0 / 2.3615933242932111574503306008555e-25 = 4234429313943308758996590.5192989


4234429313943308758996590.5192989 * 0.00000000000000000000000021381376 = 0.90537925306843927260199484611165

4234429313943308758996590.5192989 * 0.00000000000000000000000009150625 = 0.38747674740902489712793176120659

4234429313943308758996590.5192989 * 0.00000000000000000000000004100625 = 0.173638067054887804798603939982


This is a nasty brown color that you described.

But look at these numbers.

No GPU can do this kind of math with that kind of precision, and sqrt() is only approximated anyway.  If this is evaluated at run-time, overflow leading to 1.0, 1.0, 1.0 (white) is obvious.



You should find another way around using such small numbers.  Even if it works for you it may not work for someone else with another version of the HLSL or GLSL compiler.

You can’t use these kinds of numbers reliably.



L. Spiro

Edited by L. Spiro, 27 December 2012 - 01:33 AM.

It is amazing how often people try to be unique, and yet they are always trying to make others be like them. - L. Spiro 2011
I spent most of my life learning the courage it takes to go out and get what I want. Now that I have it, I am not sure exactly what it is that I want. - L. Spiro 2013
I went to my local Subway once to find some guy yelling at the staff. When someone finally came to take my order and asked, “May I help you?”, I replied, “Yeah, I’ll have one asshole to go.”
L. Spiro Engine: http://lspiroengine.com
L. Spiro Engine Forums: http://lspiroengine.com/forums

#4 Yann ALET   Members   -  Reputation: 224


Posted 27 December 2012 - 06:47 PM

Thank you both for your help !
Just so you know, the shader I'm porting is for a sky with atmosphere light scatering, and this thing plays with extremely small values.

@Zaoshi Kaba : The lambda * lambda * lambda * lambda trick worked .. ish ^^ It indeed gives me a white color, but the final effect is still broken compared to the original one in GLSL, so there is something fishy with this approach too :-/

@L. Spiro : I guess you're right, i will try to work around this.

Thanks !

Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.