Sign in to follow this  
theZapper

HLSL if statement problem

Recommended Posts

theZapper    150
Hello peeps I'm fairly new to HLSL and am having a problem with an if statement. I'm writing a simple shader to copy a texture pixel by pixel but ignore certain colours. Kind of like stencilling.
sampler2D Texture0;

float4 ps_main(float2 tex : TEXCOORD0) : COLOR0
{  
float4 white = float4(1.0f, 1.0f, 1.0f, 1.0f);
float4 clear = float4(0.0f, 0.0f, 0.0f, 0.0f);
 
   float4 val = tex2D(Texture0, tex);
   
   if (val == white)
   {
      return val;
   }
   else
   {     
      return clear;
   } 
}
However, I get an error here saying "error X3019: if statement conditional expressions must evaluate to a scalar". It would seem I can't do a direct comparison using == on two float4 variables. If works fine if I do
if(val.r == white.r)
but this is obviously only checking the red component and no good. Cheers

Share this post


Link to post
Share on other sites
theZapper    150
I tried that and it takes the instruction count from 5 up to 15 or something. If that's the only way I'll do it but I thought there must be a better way.

Share this post


Link to post
Share on other sites
remigius    1172
You could puzzle something together with other (more exotic) intrinsics, but I doubt you're going to get less than 5 instructions with that. This looks nice and clean though:

[source lang=hlsl]

float4 val = tex2D(Texture0, tex);

// reduce to integer, so components < 1 will become 0
val = trunc(val);

// all checks for non-zero, so trunc and
// then all should mean it was white

if (all(val))
{
// white
}
else
{
// non-white
}




(more on trunc, all)

This only works for white out of the box though (if it works at all). To compare to other colors, you'll probably need some subtracts but the all (or any) intrinsics may still prove useful. Oh well, for the heck of it:

[source lang=hlsl]

float4 val = tex2D(Texture0, tex);
float4 color = float4(1, 1, 1, 1);

// if they match, diff should be all 0
float4 diff = val - color;

if (any(diff))
{
// no match
}
else
{
// match
}


Share this post


Link to post
Share on other sites
hughel    126
You could do this:


if (val.r == white.r && val.g == white.g && val.b == white.b && val.a == white.a)
{
return white;
}
else
{
return clear;
}

Share this post


Link to post
Share on other sites
why hasn't anyone suggested the most obvious?

float4 val = tex2D(Texture0, tex);
float grey = val; //becomes 8-bit greyscale..note: this doesnt just take the r value.. it takes the rgb :)

if (grey == 1)
{
.....
}

//now you can easily compare to any 8 bit value (not colors unfortunately)

...incidentally I didnt think HLSL allowed different return paths yet..am I wrong?

Share this post


Link to post
Share on other sites
theZapper    150
hughel, yeah, but that makes the instruction count triple nearly :-o

I've actually done it like this now, good idea Dave! Thanks.

float4 ps_main(float2 tex : TEXCOORD0) : COLOR0
{
float4 black = float4(0.0f, 0.0f, 0.0f, 1.0f);
float4 clear = float4(0.0f, 0.0f, 0.0f, 0.0f);

float4 val = tex2D(Texture0, tex);

if (any(black.rgb + val.rgb))
{
return val;
}
else
{
return clear;
}
}


I use black as the key now, so I just add the texel to black and if any component is non-zero it passes the texel through, otherwise sets it transparent. And the instruction count is down to 4! :-D

Cheers guys.

Share this post


Link to post
Share on other sites
tweduk    144
Perhaps the == operator is considered a vector operator, where the result is a vector of 4 booleans. An 'if' statement expects a single boolean.

I haven't tried this myself, but have you tried

if (all(val == white)) {
...
} else {
...
}

Note that using == with float arguments is generally discouraged. Limited arithmetic precision makes it a dodgy practice, although it is OK in some situations.

Have you have analyzed your design in order to be certain that the values you get from the texture will be exact, and not modified by interpolation or any kind of filtering?

Share this post


Link to post
Share on other sites
remigius    1172
Quote:
Original post by theZapper
I've actually done it like this now, good idea Dave! Thanks.


Dave!?! Why oh why am I even writing code snippets anymore [headshake] [smile]


Quote:
I use black as the key now, so I just add the texel to black and if any component is non-zero it passes the texel through, otherwise sets it transparent. And the instruction count is down to 4! :-D


If black is your key now you could just use any(sampledTexel) directly, since adding that float4(0,0,0,0) doesn't really do anything. However, with your instruction count down to 4, I guess the HLSL compiler already optimized that out.

Share this post


Link to post
Share on other sites
theZapper    150
Sorry Regimus, credit goes to you for pointing me in the direction of all/any, and Dave to the addition/subtraction idea!


Everyone's a winner!

EDIT: Changed it to any(val.rgb) as suggested. Still 4 instructions but looks neater. Thanks.

Share this post


Link to post
Share on other sites
remigius    1172
Quote:
Original post by theZapper
Sorry Regimus, credit goes to you for pointing me in the direction of all/any, and Dave to the addition/subtraction idea!


Everyone's a winner!

EDIT: Changed it to any(val.rgb) as suggested. Still 4 instructions but looks neater. Thanks.


Hehe, no problem [smile]

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this