Sign in to follow this  

problem with using stencil

This topic is 3837 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

while I'm attempting to make a shadow volume demo, I'm experimenting how the stencil buffer works. I think I have understood the concept but my code doesnt work. the basic algorithm: 1: render a terrain with the natural light on, 2: use the stencil technique to mask part of the terrain by drawing an object (which will be replaced by the shadow volume) on the terrain. 3: update the fragments of the terrain which are inside of the object with the shadow light on. the goal is to render a terrain with some part in shadow.
void render_frame(void)
{
   
	d3ddev->Clear(0, NULL, D3DCLEAR_TARGET|D3DCLEAR_ZBUFFER, D3DCOLOR_XRGB(0, 0, 0), 1.0f, 0);
	d3ddev->Clear(0, NULL, D3DCLEAR_STENCIL, D3DCOLOR_XRGB(0, 0, 0), 1.0f, 0);



    d3ddev->BeginScene();

	// assume the #1 light is the natural light, #0 is the light used for drawing shadowed region.

	// *********************** pass 1  ***********************
	// pass 1
	d3ddev->SetRenderState(D3DRS_ZFUNC, D3DCMP_LESS); 
	d3ddev->SetRenderState(D3DRS_ZENABLE,TRUE);

	d3ddev->SetRenderState(D3DRS_STENCILENABLE,FALSE);
	d3ddev->SetRenderState(D3DRS_STENCILFUNC,D3DCMP_ALWAYS);
	d3ddev->SetRenderState(D3DRS_LIGHTING, TRUE);
	d3ddev->LightEnable(1, TRUE);  // turn on the normal light
	d3ddev->LightEnable(0, FALSE);// turn off the shadow light

	mesh_landscape->UpdateAll(); // update the matrices
	mesh_landscape->draw();

	objects[0]->UpdateAll(); // draw some random object
	objects[0]->draw();
	// *********************** shadow volume ***********************

	
		d3ddev->SetRenderState(D3DRS_STENCILENABLE,TRUE);
		d3ddev->SetRenderState(D3DRS_LIGHTING, FALSE); // not bothering with light when drawing the shadow volume.

		d3ddev->SetRenderState(D3DRS_ZENABLE,TRUE);
		d3ddev->SetRenderState(D3DRS_ZFUNC,D3DCMP_LESS);

		d3ddev->SetRenderState(D3DRS_ZWRITEENABLE, FALSE);

		
		d3ddev->SetRenderState( D3DRS_COLORWRITEENABLE , FALSE); // turn off the color buffer.

		// stencil configurations
		
		d3ddev->SetRenderState(D3DRS_STENCILFUNC,        D3DCMP_ALWAYS); // stencil test always passes
		
		objects[1]->UpdateAll();

		d3ddev->SetRenderState(D3DRS_STENCILPASS,       D3DSTENCILOP_INCR);
		d3ddev->SetRenderState(D3DRS_CULLMODE,D3DCULL_CCW); // render the front face of the shadow volume.
		objects[1]->draw(); // draw the front side of the shadow volume
		d3ddev->SetRenderState(D3DRS_STENCILPASS,       D3DSTENCILOP_DECR);
		d3ddev->SetRenderState(D3DRS_CULLMODE,D3DCULL_CW); // render the back of the shadow volume.
		objects[1]->draw();// draw the back side of the shadow volume.
		d3ddev->SetRenderState(D3DRS_CULLMODE,D3DCULL_CCW); // back to normal face culling.


		d3ddev->SetRenderState( D3DRS_COLORWRITEENABLE , 0xff); // turn the color buffer back on.
	
	// *********************** pass 2  ***********************
	// pass 2: draw the region that is covered by shadow.
	
		

		d3ddev->SetRenderState(D3DRS_LIGHTING, TRUE); // turn the light back on, but only the shadow light source to draw the shadowed region.
		d3ddev->LightEnable(0, TRUE);  // turn on the shadow light
		d3ddev->LightEnable(1, FALSE);// turn off the normal light

		d3ddev->SetRenderState(D3DRS_ZWRITEENABLE, TRUE); // turn z-buffer write back on.
		d3ddev->SetRenderState(D3DRS_ZENABLE,TRUE);
		
		d3ddev->SetRenderState(D3DRS_ZFUNC, D3DCMP_EQUAL);

		// stencil configurations
		//d3ddev->SetRenderState(D3DRS_STENCILENABLE,      TRUE);
		d3ddev->SetRenderState(D3DRS_STENCILFUNC,        D3DCMP_NOTEQUAL); // for this part even i change it to D3DCMP_NEVER everything still gets drawn
		d3ddev->SetRenderState(D3DRS_STENCILREF,         0x0);

		d3ddev->SetRenderState(D3DRS_STENCILFAIL,        D3DSTENCILOP_KEEP);
		d3ddev->SetRenderState(D3DRS_STENCILZFAIL,       D3DSTENCILOP_KEEP);
		d3ddev->SetRenderState(D3DRS_STENCILPASS,        D3DSTENCILOP_KEEP);
		
		// redraw the landscape
		mesh_landscape->draw();


	/*************************** end  *********************/


    d3ddev->EndScene(); 

    d3ddev->Present(NULL, NULL, NULL, NULL);

    return;
}

the result is the whole terrain mesh is redrawn in the second pass. it seems the stencil testing doesnt reject the pixels that have a zero value. I have spent a ridiculous amount of time on this already. I greatly appreciate it for reading this.

Share this post


Link to post
Share on other sites

This topic is 3837 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this