Stenciling with the stencil buffer

Started by
2 comments, last by LEPT0N 17 years, 8 months ago
Right now I have a part of the GUI in my game render a neat little object in the corner of the screen. the effect I want is that I specify a texture and draw it on a GL_QUAD in the corner of the screen. Wherever that texture's alpha is nonzero then, is where I want to draw the other stuff, so it's like you're looking through an arbitrarily shaped window. Here's my code for defining the stencil shape and then drawing:


// do this every frame I draw the GUI

glClear(GL_STENCIL_BUFFER_BIT);
glColorMask(0,0,0,0);
glEnable(GL_STENCIL_TEST);
glStencilFunc(GL_ALWAYS,1,1);
glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);
glEnable(GL_ALPHA_TEST);
glAlphaFunc(GL_GEQUAL,0.1f);

// draw the mask to define where to draw in. This texture defines the stencil shape
glBindTexture(GL_TEXTURE_2D, theMaskTexture);
glVertexPointer(2, GL_FLOAT, 0, vertexArray);
glTexCoordPointer(2, GL_INT, 0, texCoordArray);
glDrawArrays(GL_QUADS,0,4);
	
// stenciling stuff
glDisable(GL_ALPHA_TEST);
glColorMask(1,1,1,1);
glStencilFunc(GL_EQUAL, 1, 1);
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);

// draw the stuff that is only to show in the stenciled area.
drawStuff();

// clean up the stenciling
glDisable(GL_STENCIL_TEST);

However, this code is incredibly slow. Is there a better (faster) way to do this? Am I doing anything wrong?
Advertisement
u could draw to the windows alpha where u want the stuff to appear + then use a src blend of DST_ALPHA
also GL_INT for texcoords?
personally ild just use a combiner (no dstalpha/stencil required)

eg here
// draw the stuff that is only to show in the stenciled area.
drawStuff();

in glsl something like
glFragColor = glTextureA.w (alpha) * whatever;
Is there a fast way to do this without custom making a shader for it? It would seem like overkill to do that.
No one have any comments on this? I'm new to the stencil buffer and alpha testing and the like, so I'm not sure If I'm doing this the best way.

This topic is closed to new replies.

Advertisement