Sign in to follow this  
karx11erx

Need help with shader program [solved]

Recommended Posts

karx11erx    100
Task: Mask off certain areas in a texture depending on a color key in an overlay texture (multi textured). The idea behind it is that certain overlays have holes that should punch through the base texture (grates sitting on a wall e.g.). I have tried to put this in a shader combining base and overlay textures, with a mask texture telling which areas to leave transparent: Fragment shader:
uniform sampler2D btmTex, topTex, maskTex;
uniform float grAlpha; //global alpha
vec4 topColor, btmColor;
float bMask; //could do without an extra variable, but for the sake of clarity ...
void main (void) 
{
bMask = texture2D (maskTex, vec2 (gl_TexCoord [2])).a;
if (bMask < 0.5)
   discard;
else {
   topColor = texture2D (topTex, vec2(gl_TexCoord [1]));
   btmColor = texture2D (btmTex, vec2(gl_TexCoord [0]));
   if(topColor.a == 0.0)
      gl_FragColor=vec4(vec3(btmColor),btmColor.a*grAlpha)*gl_Color; 
   else
      gl_FragColor = vec4 (vec3 (mix (btmColor, topColor, topColor.a)), (btmColor.a + topColor.a) * grAlpha) * gl_Color;
   }
}





Vertex shader:
void main (void)
{
gl_TexCoord [0] = gl_MultiTexCoord0;
gl_TexCoord [1] = gl_MultiTexCoord1;
gl_TexCoord [2] = gl_MultiTexCoord2;
gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;
gl_FrontColor = gl_Color;
}





The mask texture has the format GL_ALPHA and the source format GL_UNSIGNED_BYTE. A value of 255 (1) means no hole, 0 means hole. Tried GL_RGBA4 too, but didn't work either. bmTex is bound to GL_TEXTURE0, topTex to GL_TEXTURE1, maskTex to GL_TEXTURE2 (my gfx card has enough texture units). To test it I have set the entire mask texture to "0". The shader should mask the entire texture off then. It doesn't though. Simple enough, one should think. The shader doesn't work though. It does something, because if I have it unconditionally discard every fragment, nothing gets rendered. Same for "if (bmMask > 0.5) discard;" I have a similar shader only working with bmTex and ovlTex and directly checking for the color key, and it works. As the color key gets messed up by the texture filtering before arriving in the shader though, the result doesn't look that good, so I would like to use a mask texture. What the heck am I doing wrong? I have no clue. [Edited by - karx11erx on October 11, 2007 5:00:46 PM]

Share this post


Link to post
Share on other sites
strtok    191
Well, I don't see bMask referenced at all in your else block. Isn't that what is supposed to control which texture to display? Also, I think you could simplify things quite a bit. Maybe try something like this:


uniform sampler2D btmTex, topTex, maskTex;
uniform float grAlpha;

void main ()
{
float bMask = texture2D(maskTex, gl_TexCoord[2].st).a;
vec4 topColor = texture2D(topTex, gl_TexCoord[1].st);
vec4 btmColor = texture2D(btmTex, gl_TexCoord[0].st);
vec4 color = mix(btmColor, topColor, bMask);
color.a *= grAlpha;
gl_FragColor = color;
}



Disclaimer: the above code not checked, so it may not compile as-is. But you should get the general idea. If bMask is 0.0, then you'll get something that is completely btmColor. If bMask is 1.0, you'll get something that is completely topColor. Otherwise, you'll get a nice smooth blend between the two on the edges of the holes, which is probably what you're looking for.

Share this post


Link to post
Share on other sites
karx11erx    100
That is exactly what I don't want. If bMask is 0, I want neither top nor bottom color. What you propose can simply be done with multi texturing and alpha 0 in the desired spots of the overlay texture.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this