HLSL if statement problem

Graphics and GPU Programming Programming

Started by theZapper May 16, 2008 05:47 AM

10 comments, last by remigius 15 years, 11 months ago

150

Author

May 16, 2008 05:47 AM

Hello peeps I'm fairly new to HLSL and am having a problem with an if statement. I'm writing a simple shader to copy a texture pixel by pixel but ignore certain colours. Kind of like stencilling.


sampler2D Texture0;

float4 ps_main(float2 tex : TEXCOORD0) : COLOR0
{  
float4 white = float4(1.0f, 1.0f, 1.0f, 1.0f);
float4 clear = float4(0.0f, 0.0f, 0.0f, 0.0f);
 
   float4 val = tex2D(Texture0, tex);
   
   if (val == white)
   {
      return val;
   }
   else
   {     
      return clear;
   } 
}

However, I get an error here saying "error X3019: if statement conditional expressions must evaluate to a scalar". It would seem I can't do a direct comparison using == on two float4 variables. If works fine if I do

if(val.r == white.r)

but this is obviously only checking the red component and no good. Cheers

---When I'm in command, every mission's a suicide mission!

dave

2,188

May 16, 2008 05:51 AM

Compare each component of the colour individually?

theZapper

150

Author

May 16, 2008 05:54 AM

I tried that and it takes the instruction count from 5 up to 15 or something. If that's the only way I'll do it but I thought there must be a better way.

---When I'm in command, every mission's a suicide mission!

remigius

1,173

May 16, 2008 06:35 AM

You could puzzle something together with other (more exotic) intrinsics, but I doubt you're going to get less than 5 instructions with that. This looks nice and clean though:

[source lang=hlsl]float4 val = tex2D(Texture0, tex);// reduce to integer, so components < 1 will become 0val = trunc(val); // all checks for non-zero, so trunc and // then all should mean it was whiteif (all(val)){   // white}else{    // non-white}

(more on trunc, all)

This only works for white out of the box though (if it works at all). To compare to other colors, you'll probably need some subtracts but the all (or any) intrinsics may still prove useful. Oh well, for the heck of it:

[source lang=hlsl]float4 val = tex2D(Texture0, tex);float4 color = float4(1, 1, 1, 1);// if they match, diff should be all 0float4 diff = val - color;if (any(diff)){   // no match}else{    // match}

Rim van Wersch [ MDXInfo ] [ XNAInfo ] [ YouTube ] - Do yourself a favor and bookmark this excellent free online D3D/shader book!

dave

2,188

May 16, 2008 06:39 AM

You could subtract one colour from another and then use all to see if the result is 0,0,0,0?

hughel

126

May 16, 2008 07:02 AM

You could do this:

if (val.r == white.r && val.g == white.g && val.b == white.b && val.a == white.a){       return white;}else{       return clear;}

Matt Aufderheide

129

May 16, 2008 07:26 AM

why hasn't anyone suggested the most obvious?

float4 val = tex2D(Texture0, tex);
float grey = val; //becomes 8-bit greyscale..note: this doesnt just take the r value.. it takes the rgb :)

if (grey == 1)
{
.....
}

//now you can easily compare to any 8 bit value (not colors unfortunately)

...incidentally I didnt think HLSL allowed different return paths yet..am I wrong?

theZapper

150

Author

May 16, 2008 07:47 AM

hughel, yeah, but that makes the instruction count triple nearly :-o

I've actually done it like this now, good idea Dave! Thanks.

float4 ps_main(float2 tex : TEXCOORD0) : COLOR0{  float4 black = float4(0.0f, 0.0f, 0.0f, 1.0f);float4 clear = float4(0.0f, 0.0f, 0.0f, 0.0f);    float4 val = tex2D(Texture0, tex);   if (any(black.rgb + val.rgb))   {      return val;   }   else   {      return clear;   } }

I use black as the key now, so I just add the texel to black and if any component is non-zero it passes the texel through, otherwise sets it transparent. And the instruction count is down to 4! :-D

Cheers guys.

---When I'm in command, every mission's a suicide mission!

tweduk

144

May 16, 2008 07:51 AM

Perhaps the == operator is considered a vector operator, where the result is a vector of 4 booleans. An 'if' statement expects a single boolean.

I haven't tried this myself, but have you tried

if (all(val == white)) {
...
} else {
...
}

Note that using == with float arguments is generally discouraged. Limited arithmetic precision makes it a dodgy practice, although it is OK in some situations.

Have you have analyzed your design in order to be certain that the values you get from the texture will be exact, and not modified by interpolation or any kind of filtering?

remigius

1,173

May 16, 2008 07:59 AM

Quote:Original post by theZapper
I've actually done it like this now, good idea Dave! Thanks.

Dave!?! Why oh why am I even writing code snippets anymore [headshake] [smile]

Quote:I use black as the key now, so I just add the texel to black and if any component is non-zero it passes the texel through, otherwise sets it transparent. And the instruction count is down to 4! :-D

If black is your key now you could just use any(sampledTexel) directly, since adding that float4(0,0,0,0) doesn't really do anything. However, with your instruction count down to 4, I guess the HLSL compiler already optimized that out.

Rim van Wersch [ MDXInfo ] [ XNAInfo ] [ YouTube ] - Do yourself a favor and bookmark this excellent free online D3D/shader book!

HLSL if statement problem

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

HLSL if statement problem

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Reticulating splines