Jump to content

  • Log In with Google      Sign In   
  • Create Account


HLSL pixel shader works with magic number (0.3333) but not with (1 / 3) why?


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
4 replies to this topic

#1 a2ps   Members   -  Reputation: 139

Like
0Likes
Like

Posted 28 November 2012 - 12:40 PM

hello,

trying to make a simple effect on a fullscreen triangle (triangle generated on the vertex shader using SV_VertexID) and on the pixel shader im trying color the triangle depending on its text coords (that is also generated on the vertex shader).

when i compare uv.x or uv.y with a magic number (like 0.3333 for example) it works just fine but when i use generic values (like 1 / 3 instead of using 0.3333) it doesnt work.

this is a simple example that shows the problem:

struct VSOutput
{
float4 position : SV_POSITION; // screen space
float2 uv  : TEXCOORD;	// [0,1] coords
};


PSOutput PS(VSOutput input)
{
PSOutput output = (PSOutput)0;
float oneThird = 1 / 3;
float twoThird = 2 / 3;
float x = input.uv.x;

if(x < oneThird)
  output.diffuse = float4(1.0, 0.0, 0.0, 1.0);
else if(x > twoThird)
  output.diffuse = float4(0.0, 0.0, 1.0, 1.0);
else
  output.diffuse = float4(0.0, 1.0, 0.0, 1.0);

return output;
}

this outputs blue for the entire fullscreen triangle (meaning that only the "else if(x > twoThird)" test works) BUT if i change from "x < oneThird" and "x > twoThird" to "x < 0.3333" and "x > 0.6666" it outputs 3 color stripes and eveything works great.

conclusion: using 0.3333 instead of 1 / 3 and using 0.6666 instead of 2 / 3 works great, the other way around it doesnt work. is there something im missing?

thanks in advance.
yet, another stupid signature..

Sponsor:

#2 Steve_Segreto   Crossbones+   -  Reputation: 1494

Like
2Likes
Like

Posted 28 November 2012 - 12:57 PM

You're missing precision and the internal representation of floating point numbers.

#3 MJP   Moderators   -  Reputation: 10632

Like
7Likes
Like

Posted 28 November 2012 - 01:18 PM

1 / 3 == 0 (integer division)
1.0f / 3.0f == 0.333333f (floating-point division)

#4 a2ps   Members   -  Reputation: 139

Like
0Likes
Like

Posted 28 November 2012 - 01:27 PM

wow i cant believe i fell for that.. well thanks a lot for the help :)
yet, another stupid signature..

#5 kubera   Members   -  Reputation: 891

Like
1Likes
Like

Posted 28 November 2012 - 11:37 PM

For such magics you would consider using relattions <, <= etc, not ==.
It is more safe for floats.




Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS