Sign in to follow this  

If statement not working in pixel shader

Recommended Posts

I'm trying to make fog of war by taking the terrain texture and the fog of war texture and multi-texturing them. It works pretty well except it seems to enter the if statement even if it isn't true

Here is the colors im using in the fog of war texture:

D3DCOLOR_XRGB(0,0,0); // for the completely dark unexplored areas
D3DCOLOR_XRGB(110,110,110); //this is the color of the seen but not visible areas, the actual color doesnt too much
D3DCOLOR_ARGB(254,255,255,255); //this what I use for the visible areas

Here is the pixel shader

sampler Terrain;
sampler Fog;

struct PS_INPUT
float2 terrain : TEXCOORD0;
float2 fog : TEXCOORD1;

struct PS_OUTPUT
vector color : COLOR0;

//zero out members of output
PS_OUTPUT output = (PS_OUTPUT)0;

//sample textures
vector t = tex2D(Terrain, input.terrain);
vector f = tex2D(Fog, input.fog);

vector o;

if(f.r != 110) // if the color isnt the gray color
o = t * f; // I combine these so I could reduce the number of if statements
o.a = f.a;
else // if it is then gray scale it

float avg = (t.r + t.g + t.b) / 3;
o.r = avg; o.g = avg; o.b = avg;

output.color = o;
return output;


So at the if(f.r != 110) it just enters it regardless. It works fine for the black and empty colors, though. Why would it do that?

Also if anybody knows a better way to do fog of war that would be helpful.

Share this post

Link to post
Share on other sites
tex2D does not give you a 0-255 integer. It gives you a floating point value where 0.0 is equivalent to integer 0, and 1.0 is equivalent to 255 (for the case of texture with 8 bits per channel). So your inequality is always going to evaluate to true, and you're never going to hit the else part of your if statement. You'd need to compare to 110/255, preferably with an epsilon so that you can avoid floating point comparison issues.

Share this post

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this