Sign in to follow this  
Palidor

Stencil Buffer Question

Recommended Posts

Palidor    145
Hi all, I was trying to use the stencil buffer to get rid of the outside areas of a mini-map i am working on. My main resource up until now was a prior thread located at http://www.gamedev.net/community/forums/topic.asp?topic_id=459771 Basically, i am setting up the stencil buffer to remember the image of a circle i try to render, and then i set the stencil buffer to render the background of the minimap, no rendering anywhere that stencil buffer is set to ignore. Here is the code for what I am trying to do: void DrawStencilCircle2D(IDirect3DDevice9* dev, CRenderNode* pNode) { dev->Clear(0, NULL, D3DCLEAR_STENCIL, D3DCOLOR_XRGB(0,0,255), 1.0f, 0x0); dev->SetRenderState(D3DRS_STENCILENABLE, true); dev->SetRenderState(D3DRS_STENCILPASS, D3DSTENCILOP_INCR); dev->SetRenderState(D3DRS_STENCILFUNC, D3DCMP_ALWAYS); dev->SetRenderState(D3DRS_COLORWRITEENABLE, 0); (Render the image that will set the stencil buffer here) dev->SetRenderState(D3DRS_STENCILPASS, D3DSTENCILOP_KEEP); dev->SetRenderState(D3DRS_STENCILFUNC, D3DCMP_EQUAL); dev->SetRenderState(D3DRS_STENCILREF, 1); dev->SetRenderState(D3DRS_COLORWRITEENABLE, D3DCOLORWRITEENABLE_ALPHA | D3DCOLORWRITEENABLE_RED | D3DCOLORWRITEENABLE_GREEN | D3DCOLORWRITEENABLE_BLUE); (Render the minimap here) dev->SetRenderState(D3DRS_STENCILENABLE, false); } The image that i am using to draw the minimap and the stencil buffer image renders fine if i comment out the stencil buffer setup calls. Also, the image used for the stencil is a png with a filled in circle, and everything else alpha'd out. Thank in advance! :-) Sean

Share this post


Link to post
Share on other sites
Adam_42    3629
It seems you forgot to mention what's going wrong / not working.

I'm assuming the problem is the masking isn't working.

If my guess is right, you probably want to use alpha testing (D3DRS_ALPHATESTENABLE, D3DRS_ALPHAREF, etc) for the first part to make sure the transparent pixels don't get stencil values written to them.

Share this post


Link to post
Share on other sites
Palidor    145
Tanks for the reply!

Yes, the mask is not throwing pixels out i am guessing. With the current code, I do not see any difference in the way it's being drawn. I will try enabling the alpha test and see if that solves anything. Will post again in a few!

Sean

Share this post


Link to post
Share on other sites
littlekid    229
You can try checking if your circle is rendering at the correct place. Just draw it on screen and see if it appears where it it suppose to. For the first part, you can set the stencil op to D3DSTENCILOP_REPLACE and set the ref value to 1.

Share this post


Link to post
Share on other sites
Palidor    145
This is what my teammate and I have tried so far, we tried a few other combinations, but are making no progress at all...

void DrawStencilCircle2D(IDirect3DDevice9* dev, CRenderNode* pNode)
{
dev->Clear(0, NULL, D3DCLEAR_STENCIL, D3DCOLOR_XRGB(0,0,255), 0.0f, 0x0);
dev->SetRenderState(D3DRS_STENCILENABLE, true);
dev->SetRenderState(D3DRS_STENCILPASS, D3DSTENCILOP_REPLACE);
dev->SetRenderState(D3DRS_STENCILFUNC, D3DCMP_ALWAYS);
dev->SetRenderState(D3DRS_STENCILREF, 1);
dev->SetRenderState(D3DRS_COLORWRITEENABLE, 0);

(Draw image to use as stencil mask)

dev->SetRenderState(D3DRS_STENCILPASS, D3DSTENCILOP_KEEP);
dev->SetRenderState(D3DRS_STENCILFUNC, D3DCMP_EQUAL);
dev->SetRenderState(D3DRS_STENCILREF, 1);
dev->SetRenderState(D3DRS_COLORWRITEENABLE, D3DCOLORWRITEENABLE_ALPHA | D3DCOLORWRITEENABLE_RED | D3DCOLORWRITEENABLE_GREEN | D3DCOLORWRITEENABLE_BLUE);

(Draw image that we want to only draw a certain part of using the stencil buffer)

dev->SetRenderState(D3DRS_STENCILENABLE, false);
}

I must admit, I personally am not very good at DirectX, and i also do not understand much about the stencil buffer. My teammate understands it somewhat, and we are researching a lot to figure this out. Basically it's rendering both images as if nothing was masked out. Any help is much appreciated, and thanks to those that have helped us so far!

Sean

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this