HLSL if statement problem

Recommended Posts

Hello peeps I'm fairly new to HLSL and am having a problem with an if statement. I'm writing a simple shader to copy a texture pixel by pixel but ignore certain colours. Kind of like stencilling.
sampler2D Texture0;

float4 ps_main(float2 tex : TEXCOORD0) : COLOR0
{
float4 white = float4(1.0f, 1.0f, 1.0f, 1.0f);
float4 clear = float4(0.0f, 0.0f, 0.0f, 0.0f);

float4 val = tex2D(Texture0, tex);

if (val == white)
{
return val;
}
else
{
return clear;
}
}

However, I get an error here saying "error X3019: if statement conditional expressions must evaluate to a scalar". It would seem I can't do a direct comparison using == on two float4 variables. If works fine if I do
if(val.r == white.r)
but this is obviously only checking the red component and no good. Cheers

Share on other sites
Compare each component of the colour individually?

Share on other sites
I tried that and it takes the instruction count from 5 up to 15 or something. If that's the only way I'll do it but I thought there must be a better way.

Share on other sites
You could puzzle something together with other (more exotic) intrinsics, but I doubt you're going to get less than 5 instructions with that. This looks nice and clean though:

[source lang=hlsl]float4 val = tex2D(Texture0, tex);// reduce to integer, so components < 1 will become 0val = trunc(val); // all checks for non-zero, so trunc and // then all should mean it was whiteif (all(val)){   // white}else{    // non-white}

(more on trunc, all)

This only works for white out of the box though (if it works at all). To compare to other colors, you'll probably need some subtracts but the all (or any) intrinsics may still prove useful. Oh well, for the heck of it:

[source lang=hlsl]float4 val = tex2D(Texture0, tex);float4 color = float4(1, 1, 1, 1);// if they match, diff should be all 0float4 diff = val - color;if (any(diff)){   // no match}else{    // match}

Share on other sites
You could subtract one colour from another and then use all to see if the result is 0,0,0,0?

Share on other sites
You could do this:

if (val.r == white.r && val.g == white.g && val.b == white.b && val.a == white.a){       return white;}else{       return clear;}

Share on other sites
why hasn't anyone suggested the most obvious?

float4 val = tex2D(Texture0, tex);
float grey = val; //becomes 8-bit greyscale..note: this doesnt just take the r value.. it takes the rgb :)

if (grey == 1)
{
.....
}

//now you can easily compare to any 8 bit value (not colors unfortunately)

...incidentally I didnt think HLSL allowed different return paths yet..am I wrong?

Share on other sites
hughel, yeah, but that makes the instruction count triple nearly :-o

I've actually done it like this now, good idea Dave! Thanks.
float4 ps_main(float2 tex : TEXCOORD0) : COLOR0{  float4 black = float4(0.0f, 0.0f, 0.0f, 1.0f);float4 clear = float4(0.0f, 0.0f, 0.0f, 0.0f);    float4 val = tex2D(Texture0, tex);   if (any(black.rgb + val.rgb))   {      return val;   }   else   {      return clear;   } }

I use black as the key now, so I just add the texel to black and if any component is non-zero it passes the texel through, otherwise sets it transparent. And the instruction count is down to 4! :-D

Cheers guys.

Share on other sites
Perhaps the == operator is considered a vector operator, where the result is a vector of 4 booleans. An 'if' statement expects a single boolean.

I haven't tried this myself, but have you tried

if (all(val == white)) {
...
} else {
...
}

Note that using == with float arguments is generally discouraged. Limited arithmetic precision makes it a dodgy practice, although it is OK in some situations.

Have you have analyzed your design in order to be certain that the values you get from the texture will be exact, and not modified by interpolation or any kind of filtering?

Share on other sites
Quote:
 Original post by theZapperI've actually done it like this now, good idea Dave! Thanks.

Dave!?! Why oh why am I even writing code snippets anymore [headshake] [smile]

Quote:
 I use black as the key now, so I just add the texel to black and if any component is non-zero it passes the texel through, otherwise sets it transparent. And the instruction count is down to 4! :-D

If black is your key now you could just use any(sampledTexel) directly, since adding that float4(0,0,0,0) doesn't really do anything. However, with your instruction count down to 4, I guess the HLSL compiler already optimized that out.

Share on other sites
Sorry Regimus, credit goes to you for pointing me in the direction of all/any, and Dave to the addition/subtraction idea!

Everyone's a winner!

EDIT: Changed it to any(val.rgb) as suggested. Still 4 instructions but looks neater. Thanks.

Share on other sites
Quote:
 Original post by theZapperSorry Regimus, credit goes to you for pointing me in the direction of all/any, and Dave to the addition/subtraction idea!Everyone's a winner!EDIT: Changed it to any(val.rgb) as suggested. Still 4 instructions but looks neater. Thanks.

Hehe, no problem [smile]

Create an account

Register a new account

• Forum Statistics

• Total Topics
627778
• Total Posts
2979026

• 11
• 10
• 10
• 23
• 9