Jump to content

  • Log In with Google      Sign In   
  • Create Account

FREE SOFTWARE GIVEAWAY

We have 4 x Pro Licences (valued at $59 each) for 2d modular animation software Spriter to give away in this Thursday's GDNet Direct email newsletter.


Read more in this forum topic or make sure you're signed up (from the right-hand sidebar on the homepage) and read Thursday's newsletter to get in the running!


Shadow volumes Z-fail


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
5 replies to this topic

#1 menohack   Members   -  Reputation: 216

Like
0Likes
Like

Posted 09 July 2013 - 12:24 PM

I have Z-pass working for my D3D11 application but no matter what I try I can't get z-fail to work. Here is my depth stencil state for building the stencil buffer with z-pass:

D3D11_DEPTH_STENCIL_DESC depthStencilDesc = {0};
	depthStencilDesc.DepthEnable = true;
	//Disable depth writes when rendering the volumes
	depthStencilDesc.DepthWriteMask = D3D11_DEPTH_WRITE_MASK_ZERO;
	depthStencilDesc.DepthFunc = D3D11_COMPARISON_LESS;
	depthStencilDesc.StencilEnable = true;
	depthStencilDesc.StencilReadMask = D3D11_DEFAULT_STENCIL_READ_MASK;
	depthStencilDesc.StencilWriteMask = D3D11_DEFAULT_STENCIL_WRITE_MASK;

	//Stencil operation for front-facing polygons should increment
	depthStencilDesc.FrontFace.StencilFunc = D3D11_COMPARISON_ALWAYS;
	depthStencilDesc.FrontFace.StencilDepthFailOp = D3D11_STENCIL_OP_KEEP;
	depthStencilDesc.FrontFace.StencilPassOp = D3D11_STENCIL_OP_INCR;
	depthStencilDesc.FrontFace.StencilFailOp = D3D11_STENCIL_OP_KEEP;
	

	//Stencil operation for back-facing polygons should decrement
	depthStencilDesc.BackFace.StencilFunc = D3D11_COMPARISON_ALWAYS;
	depthStencilDesc.BackFace.StencilDepthFailOp = D3D11_STENCIL_OP_KEEP;
	depthStencilDesc.BackFace.StencilPassOp = D3D11_STENCIL_OP_DECR;
	depthStencilDesc.BackFace.StencilFailOp = D3D11_STENCIL_OP_KEEP;
	

	hr = g_pd3dDevice->CreateDepthStencilState(&depthStencilDesc, &buildShadowVolumeDepthStencilState);

How would I modify this to do z-fail?



Sponsor:

#2 mhagain   Crossbones+   -  Reputation: 8284

Like
1Likes
Like

Posted 09 July 2013 - 12:47 PM

For Z-fail you need this:

//Stencil operation for front-facing polygons should decrement
depthStencilDesc.FrontFace.StencilFunc = D3D11_COMPARISON_ALWAYS;
depthStencilDesc.FrontFace.StencilDepthFailOp = D3D11_STENCIL_OP_DECR;
depthStencilDesc.FrontFace.StencilPassOp = D3D11_STENCIL_OP_KEEP;
depthStencilDesc.FrontFace.StencilFailOp = D3D11_STENCIL_OP_KEEP;

//Stencil operation for back-facing polygons should increment
depthStencilDesc.BackFace.StencilFunc = D3D11_COMPARISON_ALWAYS;
depthStencilDesc.BackFace.StencilDepthFailOp = D3D11_STENCIL_OP_INCR;
depthStencilDesc.BackFace.StencilPassOp = D3D11_STENCIL_OP_KEEP;
depthStencilDesc.BackFace.StencilFailOp = D3D11_STENCIL_OP_KEEP;

This was reverse-engineered from Doom 3 (via GL Intercept); Doom 3's depth-pass setup matches yours so you can expect this to work.  Beware the patent, of course.


It appears that the gentleman thought C++ was extremely difficult and he was overjoyed that the machine was absorbing it; he understood that good C++ is difficult but the best C++ is well-nigh unintelligible.


#3 menohack   Members   -  Reputation: 216

Like
0Likes
Like

Posted 09 July 2013 - 01:00 PM

That makes sense but it doesn't work. It makes the floor black except where geometry is behind the floor. Don't I need to change DepthFunc to GREATER? I want only fragments behind the depth map to be counted. Even when I try that it doesn't work.



#4 mhagain   Crossbones+   -  Reputation: 8284

Like
1Likes
Like

Posted 09 July 2013 - 01:55 PM

The other thing I'm seeing in the Doom 3 code is that you also need to disable backface culling for both Z-pass and Z-fail.  I'm not seeing anything about switching the depth test.  It's worth noting that Doom 3 uses a stencil ref of 128, mask 255 and clears the stencil buffer to 128, so you may not already have those.  You may also prefer to use D3D11_STENCIL_OP_INCR_SAT and D3D11_STENCIL_OP_DECR_SAT in all cases.


It appears that the gentleman thought C++ was extremely difficult and he was overjoyed that the machine was absorbing it; he understood that good C++ is difficult but the best C++ is well-nigh unintelligible.


#5 menohack   Members   -  Reputation: 216

Like
0Likes
Like

Posted 09 July 2013 - 04:37 PM

Yeah I'm doing all of those things. Like I said, Z-pass works. That implies that I have culling disabled. I use a stencil ref of 127 and clear to that as well. It may be worth using the saturate option for the stencil operation but it shouldn't be a problem for my simple test case.

 

I've been trying to get this for weeks and I just can't figure it out. There has to be something going on behind the scenes that I'm missing.

 

I've attached a picture of Z-pass working and what happens when I try z-fail using StencilDepthFailOp = INCR/DECR and DepthFunc = LESS. Note that StencilDepthFailOp is KEEP for z-pass and StencilPassOp is INCR/DECR instead.

 

The third image is when I set DepthFunc to GREATER_EQUAL and StencilDepthFailOp to INCR/DECR. This should be another method of z-pass because I have inverted the depth function. Now values that are less than the depth value will fail the depth test and cause the stencil value to change. But I can't explain the squares of grey pixels around the chair. Is D3D doing the depth test at a lower resolution, then using that decision for a whole square of fragments? I can't think of any other reason why those little grey squares would appear.

 

One place where I am confused by the API: what happens if the depth AND the stencil tests fail? There is a case for the depth fail operation and a case for the stencil fail operation. Which one occurs?

 

I appreciate the replies mhagain.

Attached Thumbnails

  • ZPass.png
  • ZFailProblem.png
  • ZPassDepthProblem.png


#6 menohack   Members   -  Reputation: 216

Like
0Likes
Like

Posted 15 July 2013 - 11:00 PM

Solved it! My geometry shader was producing front caps and back caps with inverted normals. This caused the stencil count to be off because front faces increment and back faces decrement.

 

For future reference, the way mhagain set up the DepthStencil is the correct way. The front-facing shadow polygons could increment and the back-facing polygons could decrement, but it doesn't matter. The depth test should be LESS, so that failures are equal to or greater than the depth buffer.

 

The accidental inverted normal is like the missed decimal place for graphics programmers: http://www.youtube.com/watch?v=IPLnYP9Oo8s






Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS