Sign in to follow this  
Capoeirista

Stencil Buffer Issue

Recommended Posts

Capoeirista    3266

Hi there,

 

I have a 2D scene with three layers - background, center and foreground. I'm rendering the central layer first so I can perform lighting calculations, and then want to render the background followed by the foreground.

 

I'm currently trying to set up a stencil buffer when rendering the central layer, so any objects drawn in the background pass will skip pixels that already have central there.


Only trouble is the stencil test doesn't seem to apply (unless I'm completely missing how it's supposed to work), the background objects always overwrite the foreground objects (as they're second).

 

I'm setting everything up like this :

 

myColourTarget = new RenderTarget2D( aGraphicsDevice, width, height, false, SurfaceFormat.Color, DepthFormat.Depth24Stencil8 );

...

graphicsDevice.SetRenderTarget( myColourTarget );
graphicsDevice.Clear( ClearOptions.Stencil | ClearOptions.Target, ClearColor, -1.0f, 0 );

myDepthStencilState.StencilEnable = true;
myDepthStencilState.StencilFunction = CompareFunction.Always;
myDepthStencilState.ReferenceStencil = 1;
myDepthStencilState.StencilPass = StencilOperation.Replace;
myDepthStencilState.DepthBufferEnable = false;

spriteBatch.Begin( SpriteSortMode.Immediate, BlendState.AlphaBlend, null, myDepthStencilState, null, objectEffect);

// draw all central objects

spriteBatch.End()

myDepthStencilState.StencilEnable = true;
myDepthStencilState.StencilFunction = CompareFunction.Equal;
myDepthStencilState.ReferenceStencil = 0;
myDepthStencilState.StencilPass = StencilOperation.Keep;
myDepthStencilState.DepthBufferEnable = false;

spriteBatch.Begin( SpriteSortMode.Immediate, BlendState.AlphaBlend, null, myDepthStencilState, null, objectEffect);

// draw all background objects

spriteBatch.End();

 

 

I've tried to do the same sort of effect using depth testing, but I can't seem to get the alpha blending correct - pixels from central objects with an alpha value of 0 always overwrite background images, and the clear colour for the render target bleeds through ( http://i.imgur.com/lRkh3Mj.png )

 

 

 

Any ideas what I'm doing wrong with the stencil test... I figured that the first stencil state would write a 1 whenever a pixel value is written, and the second state would set things up sot that it will only keep the pixel if there's a 0 in the stencil buffer.

 

Cheers!

Edited by Orangeatang

Share this post


Link to post
Share on other sites
mhagain    13430

The reason why that didn't work for you was because state objects in D3D10+ are immutable, once created with their initial states they can't be modified, so you must either destroy and recreate the object with new states (not recommended as runtime destruction/creation is expensive) or create multiple state objects.  This can seem like a bit of a pain-in-the-ass when you first encounter it, but in theory it should give you higher performance as (1) a whole bunch of state can be set in a single API call, and (2) hardware groups state internally in a similar manner anyway (in practice the high optimization of D3D9 drivers over more than a decade, coupled with lazy state changes in the driver, means that you may not actually get that increased performance - oh well).

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this