Sign in to follow this  

Stencil Buffer Usage Clarification

Recommended Posts

Hello, please help me understand how to use the Stencil Buffer. I have tried some things and I can't seem to make it work for me at all so I've gone back to basics just to figure things out. Right now in my pixel shader i am just returning red: float4(1,0,0,1); and i have these things set for the render states set before the draw call: device->Clear(0, NULL, D3DCLEAR_STENCIL, D3DCOLOR_ARGB(0,0,0,0), 1.0, 0); device->SetRenderState(D3DRS_STENCILENABLE, true); device->SetRenderState(D3DRS_STENCILFUNC, D3DCMP_EQUAL); device->SetRenderState(D3DRS_STENCILREF, 2); So my assumption is that when i do this, i should not see any of that red from the Pixel Shader, but the entire screen is red. I can change the value for D3DRS_STENCILFUNC to anything and it will still show the red on the screen. Can anybody give me some guidance or a link to understand how to make use of the render states pertaining to the stencil buffer? Thanks

Share this post

Link to post
Share on other sites
here is the create call of my depth stencil buffer:
device->CreateDepthStencilSurface(screenWidth, screenHeight, D3DFMT_D24S8,
D3DMULTISAMPLE_NONE, 0, true, &dsb, NULL);

Share this post

Link to post
Share on other sites
i did, here are the values:

ZeroMemory(&d3dpp, sizeof(d3dpp));
d3dpp.Windowed = true;
d3dpp.hDeviceWindow = hWnd;
d3dpp.BackBufferFormat = D3DFMT_A8R8G8B8;
d3dpp.BackBufferWidth = SCREEN_WIDTH;
d3dpp.BackBufferHeight = SCREEN_HEIGHT;
d3dpp.EnableAutoDepthStencil = TRUE;
d3dpp.AutoDepthStencilFormat = D3DFMT_D24S8;

Share this post

Link to post
Share on other sites
I am prety noob at shaders (and directx actually :p). So I might misguide you, sorry.

When I use stencil buffer and shaders, I define my stencil test variables in shader. Like

[sourcecode]    pass P1
VertexShader = compile vs_1_1 VS();
PixelShader = compile ps_2_0 PS2();
ZEnable = false;
ZWriteEnable = false;


Maybe device->SetRenderState variables are disabled while using shaders or something? Again I am also new so.. =)

Share this post

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this