Sign in to follow this  

Stencil Buffer Usage Clarification

This topic is 3038 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hello, please help me understand how to use the Stencil Buffer. I have tried some things and I can't seem to make it work for me at all so I've gone back to basics just to figure things out. Right now in my pixel shader i am just returning red: float4(1,0,0,1); and i have these things set for the render states set before the draw call: device->Clear(0, NULL, D3DCLEAR_STENCIL, D3DCOLOR_ARGB(0,0,0,0), 1.0, 0); device->SetRenderState(D3DRS_STENCILENABLE, true); device->SetRenderState(D3DRS_STENCILFUNC, D3DCMP_EQUAL); device->SetRenderState(D3DRS_STENCILREF, 2); So my assumption is that when i do this, i should not see any of that red from the Pixel Shader, but the entire screen is red. I can change the value for D3DRS_STENCILFUNC to anything and it will still show the red on the screen. Can anybody give me some guidance or a link to understand how to make use of the render states pertaining to the stencil buffer? Thanks

Share this post


Link to post
Share on other sites
here is the create call of my depth stencil buffer:
device->CreateDepthStencilSurface(screenWidth, screenHeight, D3DFMT_D24S8,
D3DMULTISAMPLE_NONE, 0, true, &dsb, NULL);

Share this post


Link to post
Share on other sites
i did, here are the values:

D3DPRESENT_PARAMETERS d3dpp;
ZeroMemory(&d3dpp, sizeof(d3dpp));
d3dpp.Windowed = true;
d3dpp.SwapEffect = D3DSWAPEFFECT_DISCARD;
d3dpp.hDeviceWindow = hWnd;
d3dpp.BackBufferFormat = D3DFMT_A8R8G8B8;
d3dpp.BackBufferWidth = SCREEN_WIDTH;
d3dpp.BackBufferHeight = SCREEN_HEIGHT;
d3dpp.EnableAutoDepthStencil = TRUE;
d3dpp.AutoDepthStencilFormat = D3DFMT_D24S8;

Share this post


Link to post
Share on other sites
I am prety noob at shaders (and directx actually :p). So I might misguide you, sorry.

When I use stencil buffer and shaders, I define my stencil test variables in shader. Like

[sourcecode]    pass P1
{
VertexShader = compile vs_1_1 VS();
PixelShader = compile ps_2_0 PS2();
ZEnable = false;
ZWriteEnable = false;

STENCILENABLE= TRUE;
STENCILFUNC = ALWAYS;
STENCILPASS = REPLACE;
STENCILREF= 0x01 ;
STENCILMASK= 0xFF ;
STENCILWRITEMASK=0xFF ;
}



Maybe device->SetRenderState variables are disabled while using shaders or something? Again I am also new so.. =)

Share this post


Link to post
Share on other sites

This topic is 3038 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this