View more

View more

View more

### Image of the Day Submit

IOTD | Top Screenshots

### The latest, straight to your Inbox.

Subscribe to GameDev.net Direct to receive the latest updates and exclusive content.

# HLSL pixel shader works with magic number (0.3333) but not with (1 / 3) why?

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

4 replies to this topic

### #1a2ps  Members

Posted 28 November 2012 - 12:40 PM

hello,

trying to make a simple effect on a fullscreen triangle (triangle generated on the vertex shader using SV_VertexID) and on the pixel shader im trying color the triangle depending on its text coords (that is also generated on the vertex shader).

when i compare uv.x or uv.y with a magic number (like 0.3333 for example) it works just fine but when i use generic values (like 1 / 3 instead of using 0.3333) it doesnt work.

this is a simple example that shows the problem:

struct VSOutput
{
float4 position : SV_POSITION; // screen space
float2 uv  : TEXCOORD;	// [0,1] coords
};

PSOutput PS(VSOutput input)
{
PSOutput output = (PSOutput)0;
float oneThird = 1 / 3;
float twoThird = 2 / 3;
float x = input.uv.x;

if(x < oneThird)
output.diffuse = float4(1.0, 0.0, 0.0, 1.0);
else if(x > twoThird)
output.diffuse = float4(0.0, 0.0, 1.0, 1.0);
else
output.diffuse = float4(0.0, 1.0, 0.0, 1.0);

return output;
}


this outputs blue for the entire fullscreen triangle (meaning that only the "else if(x > twoThird)" test works) BUT if i change from "x < oneThird" and "x > twoThird" to "x < 0.3333" and "x > 0.6666" it outputs 3 color stripes and eveything works great.

conclusion: using 0.3333 instead of 1 / 3 and using 0.6666 instead of 2 / 3 works great, the other way around it doesnt work. is there something im missing?

yet, another stupid signature..

### #2Steve_Segreto  Members

Posted 28 November 2012 - 12:57 PM

You're missing precision and the internal representation of floating point numbers.

### #3MJP  Moderators

Posted 28 November 2012 - 01:18 PM

POPULAR

1 / 3 == 0 (integer division)
1.0f / 3.0f == 0.333333f (floating-point division)

### #4a2ps  Members

Posted 28 November 2012 - 01:27 PM

wow i cant believe i fell for that.. well thanks a lot for the help
yet, another stupid signature..

### #5kubera  Members

Posted 28 November 2012 - 11:37 PM

For such magics you would consider using relattions <, <= etc, not ==.
It is more safe for floats.

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.