Jump to content
  • Advertisement
Sign in to follow this  

problem with using stencil

This topic is 4125 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

while I'm attempting to make a shadow volume demo, I'm experimenting how the stencil buffer works. I think I have understood the concept but my code doesnt work. the basic algorithm: 1: render a terrain with the natural light on, 2: use the stencil technique to mask part of the terrain by drawing an object (which will be replaced by the shadow volume) on the terrain. 3: update the fragments of the terrain which are inside of the object with the shadow light on. the goal is to render a terrain with some part in shadow.
void render_frame(void)
	d3ddev->Clear(0, NULL, D3DCLEAR_TARGET|D3DCLEAR_ZBUFFER, D3DCOLOR_XRGB(0, 0, 0), 1.0f, 0);
	d3ddev->Clear(0, NULL, D3DCLEAR_STENCIL, D3DCOLOR_XRGB(0, 0, 0), 1.0f, 0);


	// assume the #1 light is the natural light, #0 is the light used for drawing shadowed region.

	// *********************** pass 1  ***********************
	// pass 1
	d3ddev->SetRenderState(D3DRS_ZFUNC, D3DCMP_LESS); 

	d3ddev->SetRenderState(D3DRS_LIGHTING, TRUE);
	d3ddev->LightEnable(1, TRUE);  // turn on the normal light
	d3ddev->LightEnable(0, FALSE);// turn off the shadow light

	mesh_landscape->UpdateAll(); // update the matrices

	objects[0]->UpdateAll(); // draw some random object
	// *********************** shadow volume ***********************

		d3ddev->SetRenderState(D3DRS_LIGHTING, FALSE); // not bothering with light when drawing the shadow volume.


		d3ddev->SetRenderState(D3DRS_ZWRITEENABLE, FALSE);

		d3ddev->SetRenderState( D3DRS_COLORWRITEENABLE , FALSE); // turn off the color buffer.

		// stencil configurations
		d3ddev->SetRenderState(D3DRS_STENCILFUNC,        D3DCMP_ALWAYS); // stencil test always passes

		d3ddev->SetRenderState(D3DRS_STENCILPASS,       D3DSTENCILOP_INCR);
		d3ddev->SetRenderState(D3DRS_CULLMODE,D3DCULL_CCW); // render the front face of the shadow volume.
		objects[1]->draw(); // draw the front side of the shadow volume
		d3ddev->SetRenderState(D3DRS_STENCILPASS,       D3DSTENCILOP_DECR);
		d3ddev->SetRenderState(D3DRS_CULLMODE,D3DCULL_CW); // render the back of the shadow volume.
		objects[1]->draw();// draw the back side of the shadow volume.
		d3ddev->SetRenderState(D3DRS_CULLMODE,D3DCULL_CCW); // back to normal face culling.

		d3ddev->SetRenderState( D3DRS_COLORWRITEENABLE , 0xff); // turn the color buffer back on.
	// *********************** pass 2  ***********************
	// pass 2: draw the region that is covered by shadow.

		d3ddev->SetRenderState(D3DRS_LIGHTING, TRUE); // turn the light back on, but only the shadow light source to draw the shadowed region.
		d3ddev->LightEnable(0, TRUE);  // turn on the shadow light
		d3ddev->LightEnable(1, FALSE);// turn off the normal light

		d3ddev->SetRenderState(D3DRS_ZWRITEENABLE, TRUE); // turn z-buffer write back on.
		d3ddev->SetRenderState(D3DRS_ZFUNC, D3DCMP_EQUAL);

		// stencil configurations
		//d3ddev->SetRenderState(D3DRS_STENCILENABLE,      TRUE);
		d3ddev->SetRenderState(D3DRS_STENCILFUNC,        D3DCMP_NOTEQUAL); // for this part even i change it to D3DCMP_NEVER everything still gets drawn
		d3ddev->SetRenderState(D3DRS_STENCILREF,         0x0);

		d3ddev->SetRenderState(D3DRS_STENCILFAIL,        D3DSTENCILOP_KEEP);
		d3ddev->SetRenderState(D3DRS_STENCILZFAIL,       D3DSTENCILOP_KEEP);
		d3ddev->SetRenderState(D3DRS_STENCILPASS,        D3DSTENCILOP_KEEP);
		// redraw the landscape

	/*************************** end  *********************/


    d3ddev->Present(NULL, NULL, NULL, NULL);


the result is the whole terrain mesh is redrawn in the second pass. it seems the stencil testing doesnt reject the pixels that have a zero value. I have spent a ridiculous amount of time on this already. I greatly appreciate it for reading this.

Share this post

Link to post
Share on other sites
Sign in to follow this  

  • Advertisement

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!