Direct writing to stencil buffer

Started by
5 comments, last by DonnieDarko 15 years, 8 months ago
I need to fill stencil portion of a D24S8 depth buffer with a custom pattern loaded from disk. How can I do it? DX9.0c, no shaders.
Advertisement
AFAIK you can't write an image directly to the stencil buffer, but I think you can fill the stencil buffer by a an image by rendering as many passes as you have different colors in your image.

1 Load your pattern into the alpha channel of a texture.
2 Disable writes to Z and Color
3 Render a full screen quad with the texture and set the AlphaFunc and AlphaRef states to reject all texels except the value you want to write.
4 Set StencilRef to the current pattern value and StencilOp to replace.

Repeat step 3 and 4 for each value in your image.

I think this an unorthodox use of the stencil buffer, are you sure you need this? If you explain what kind of effect you are trying to achieve, there may be someone who can explain a better way to do it.
1. Load the pattern into the alpha channel of a texture (e.g. D3DFMT_A8).

2. When you Clear(), ensure stencil is cleared.

3. Set the stencil render states to:
a) write to the stencil buffer, and
b) always pass tests against the stencil buffer

4. Set the alpha test render states to only pass pixels whose alpha is greater than 0 (assuming a 1-bit mask pattern).

5. Disable colour writes (D3DRS_COLORWRITEENABLE).

6. Draw a fullscreen quad with the texture applied.

7. Set appropriate states and draw the things you want to be masked.


Explanation: a pixel that fails alpha test won't be written to (or tested against) the stencil buffer or depth buffer.

Simon O'Connor | Technical Director (Newcastle) Lockwood Publishing | LinkedIn | Personal site

Those render states are killing me...

The problem: I have a D3DFMT_A8 texture. I wan't to setup device so than when next quad is rendered with that texture only texels with a specific alpha value are passed to stencil buffer test. I want to set D3DRS_ALPHAREF to that alpha value. I don't want to render that quad, just use alpha tests, to trigger stencil test, to fill stencil buffer. Pls, help.

Damn, I really would like to work with HLSL instead of this messy !@#$ of render/texture states/stages...
Quote:Original post by Turold
I don't want to render that quad


You have to render something, otherwise you can't set any values in the stencil buffer. No way around that.

Quote:Original post by Turold
Damn, I really would like to work with HLSL instead of this messy !@#$ of render/texture states/stages...


Well you can embed all of those render states in your Effect file via effect states, if that helps...

technique FillStencil{	pass p0	{			StencilEnable = TRUE;		StencilFunc = ALWAYS;		StencilPass = REPLACE;		StencilRef = 1;		AlphaTestEnable = TRUE;		AlphaRef = 0;                ColorWriteEnable = 0;			// compile VS and PS			}}



Quote:Original post by MJP
Quote:Original post by Turold
I don't want to render that quad


You have to render something, otherwise you can't set any values in the stencil buffer. No way around that.


Ok, I can accept rendering that quad. Just give me the code for setting up textures stage 0 and render states to do the job neatly. How should I approach rendering D3DFMT_A8 textured quad? Using diffuse vertex color I suppose? If I could just lock that damn stencil buffer. I know what I want to do with that pixels, but just can't express in fixed pipeline "language". -_-

[Edited by - Turold on July 31, 2008 11:29:04 AM]
Don't worry about diffuse or other lighting related stuff. Just setup a quad with only position and texture coordinates in the vertex. Something like this:

float verts[4*5] = {-1, 1, 0, 0, 0, 1, 1, 0, 1, 01,-1, 0, 1, 1-1,-1, 0, 0, 1};unsigned short indices[6] = {0,1,2, 2,3,0};DrawIndexedPrimitiveUP(D3DPT_TRIANGLELIST, 0, 4, 2, indices, D3DFMT_INDEX16, verts, 4*5);


The TextueStageState needs to be correct for D3DTSS_ALPHAOP and D3DTSS_ALPHAARG1 and the the default values seems to be correct for you, but you could set them explicitly just to be sure. You don't need to worry about texture states for color operations since color writes are disabled.

This topic is closed to new replies.

Advertisement