Sign in to follow this  
RonD

Stencil Buffer not working as expected

Recommended Posts

Hi, the following piece of code is meant to render panels of a GUI. The idea is that you can have nested panels where only those parts of the child panel are drawn which are inside of the parent panel. My idea was to have distinct stencil ref values for each layer if you will. Say panel b is within a, then the stencil buffer would initially be set to zero. Then as panel a is drawn, the values will be increased to 1 wherever panel a is drawn. Panel b would further increase the value by 1 but would only draw itself at places where the stencil value is already one. Makes sense? Anyway, here is what i tried:
void Panel::draw(){
	Widget::draw();

	GLint stencilRefContainer = 0;
	glGetIntegerv(GL_STENCIL_REF,&stencilRefContainer);

	GLint stencilRefInc = stencilRefContainer + 1; // reference value where container widget was drawn

	for(std::vector<Widget*>::iterator i = Container::begin();
		i!=Container::end();
		i++){

			// recreate original stencil mask for each child:	
			glColorMask(false,false,false,false);
				// flatten nested layers
				glStencilFunc(GL_GEQUAL,stencilRefContainer,0xFFFFFFFF);
				glStencilOp(GL_KEEP, GL_REPLACE, GL_REPLACE);
				Widget::draw();

				// increment SRC where widget is drawn -> where the current container widget is drawn stencilref will be SRC++ = SRI
				glStencilFunc(GL_EQUAL,stencilRefContainer,0xFFFFFFFF);
				glStencilOp(GL_KEEP, GL_INCR, GL_INCR);
				Widget::draw();	

			// draw each widget using that mask:
			glColorMask(true,true,true,true);
				Widget::pushMatrix(); // do relative coordinate transforms
					glStencilFunc(GL_EQUAL, stencilRefInc, 0xFFFFFFFF); // draw children where ref indicates parent widget
					glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP); 
					(*i)->draw();
				Widget::popMatrix(); // undo transforms
	}
	
	// reset to what it was previously:
	glStencilFunc(GL_ALWAYS,stencilRefContainer,0xFFFFFFFF);
}


I made three test setups of nested panels. The blue one is the root panel, the white one is a child of the blue one and the red one is a child of the white one. Here is what I get: Stencil Panel Fail In the bottom left, the red panel is visible outside the blue panel, which is not what I want. What did I do wrong? How can I solve the problem? Thank you, RonD

Share this post


Link to post
Share on other sites
yes, I did. Before the call to the recursive method shown, I do all this to initialise the stencil buffer etc:

glClearStencil(0);
glClear(GL_STENCIL_BUFFER_BIT);
glStencilFunc(GL_ALWAYS,0,0xFFFFFFFF);
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
glEnable(GL_STENCIL_TEST);

Share this post


Link to post
Share on other sites
the problem is its drawing on the black, see what stencil value this is

try something like to visualize what the current stencil buffer contains

VEC4 stencil_cols[] = {
VEC4( 0.00, 0.00, 0.00, 1.0 ),
VEC4( 0.00, 0.00, 0.50, 1.0 ),
VEC4( 0.00, 0.00, 1.00, 1.0 ),
VEC4( 0.00, 0.50, 0.00, 1.0 ),
VEC4( 0.00, 1.00, 0.00, 1.0 ),
VEC4( 0.50, 0.00, 0.00, 1.0 ),
VEC4( 1.00, 0.00, 0.00, 1.0 ),
VEC4( 1.00, 0.00, 0.50, 1.0 ),
VEC4( 1.00, 0.00, 1.00, 1.0 ),
VEC4( 1.00, 1.00, 0.50, 1.0 ),
VEC4( 0.10, 0.10, 0.10, 1.0 ),
VEC4( 0.20, 0.20, 0.20, 1.0 ),
VEC4( 0.30, 0.30, 0.30, 1.0 ),
VEC4( 0.40, 0.40, 0.40, 1.0 ),
VEC4( 0.50, 0.50, 0.50, 1.0 ),
VEC4( 0.60, 0.60, 0.60, 1.0 ),
VEC4( 0.70, 0.70, 0.70, 1.0 ),
VEC4( 0.80, 0.80, 0.80, 1.0 ),
VEC4( 0.90, 0.90, 0.90, 1.0 ),
VEC4( 1.00, 1.00, 1.00, 1.0 ),
VEC4( 1.00, 1.00, 0.00, 1.0 )};

GLstenciltest stencil;
stencil.setIdentity();
stencil.enable_stenciltest = true;
stencil.stencil_fail = GL_KEEP;
stencil.stencil_pass_depth_fail = GL_KEEP;
stencil.stencil_pass_depth_pass = GL_KEEP;

for ( int i=0; i<20; i++ )
{
stencil.stencil_func = GL_EQUAL;
stencil.stencil_valuemask = 0xff;
stencil.stencil_ref = i;
g_rm.ogl.setStencil( stencil );

glColor4fv( stencil_cols[i] );
glBegin( GL_QUADS );
glVertex3f( -1,-1, 0 );
glVertex3f( 1,-1, 0 );
glVertex3f( 1, 1, 0 );
glVertex3f( -1, 1, 0 );
glEnd();
}

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this