Jump to content
  • Advertisement
Sign in to follow this  
beatlefan

If statement not working in pixel shader

This topic is 2613 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I'm trying to make fog of war by taking the terrain texture and the fog of war texture and multi-texturing them. It works pretty well except it seems to enter the if statement even if it isn't true

Here is the colors im using in the fog of war texture:

D3DCOLOR_XRGB(0,0,0); // for the completely dark unexplored areas
D3DCOLOR_XRGB(110,110,110); //this is the color of the seen but not visible areas, the actual color doesnt too much
D3DCOLOR_ARGB(254,255,255,255); //this what I use for the visible areas

Here is the pixel shader



sampler Terrain;
sampler Fog;

struct PS_INPUT
{
float2 terrain : TEXCOORD0;
float2 fog : TEXCOORD1;
};

struct PS_OUTPUT
{
vector color : COLOR0;
};


PS_OUTPUT Main(PS_INPUT input)
{
//zero out members of output
PS_OUTPUT output = (PS_OUTPUT)0;

//sample textures
vector t = tex2D(Terrain, input.terrain);
vector f = tex2D(Fog, input.fog);

vector o;

if(f.r != 110) // if the color isnt the gray color
{
o = t * f; // I combine these so I could reduce the number of if statements
o.a = f.a;
}
else // if it is then gray scale it
{

float avg = (t.r + t.g + t.b) / 3;
o.r = avg; o.g = avg; o.b = avg;
}

output.color = o;
return output;

}



So at the if(f.r != 110) it just enters it regardless. It works fine for the black and empty colors, though. Why would it do that?

Also if anybody knows a better way to do fog of war that would be helpful.

Share this post


Link to post
Share on other sites
Advertisement
tex2D does not give you a 0-255 integer. It gives you a floating point value where 0.0 is equivalent to integer 0, and 1.0 is equivalent to 255 (for the case of texture with 8 bits per channel). So your inequality is always going to evaluate to true, and you're never going to hit the else part of your if statement. You'd need to compare to 110/255, preferably with an epsilon so that you can avoid floating point comparison issues.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!