Jump to content
  • Advertisement
Sign in to follow this  
webjeff

Stencil Shadow Problem Please Look (Screenie)

This topic is 5401 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Please look at this: http://www.webjeff.com/img/newshadow.jpg I have no idea how to fix it, I can't figure this out. Has anyone else seen anything like this before or why it may happen like that??? Thanks Jeff.

Share this post


Link to post
Share on other sites
Advertisement
Guest Anonymous Poster
how do you determine the winding of verts for the extruded faces?

Share this post


Link to post
Share on other sites
I bet it doesn't happen if you use a utility sphere like one created from D3DXCreateSphere. Check to see if this is realy the case. If so most likely your models aren't closed well. They might look ok but the triangles that make up the model don't really touch each other. Just fix the model.

Share this post


Link to post
Share on other sites
Are you using the normals and the dot product to only extrude triangles whos front faces are facing the light?

How far are you extruding the volumes? I had a problem like this and I figured the bottom cap of the shadow volume must of been going past the far clip plane, try lowering the extrusion distance to just more than will ever be needed.

Share this post


Link to post
Share on other sites
I just go through my indicies starting with 0.


Here is my ShadowVolume Code:


#include "stdafx.h"
#include "ShadowVolume.h"

//-----------------------------------------------------------------------------
// Name: render
// Desc:
//-----------------------------------------------------------------------------
HRESULT ShadowVolume::render( LPDIRECT3DDEVICE9 pd3dDevice )
{

pd3dDevice->SetFVF( D3DFVF_XYZ );

return pd3dDevice->DrawPrimitiveUP( D3DPT_TRIANGLELIST, m_dwNumVertices/3,
m_pVertices, sizeof(D3DXVECTOR3) );
}

//-----------------------------------------------------------------------------
// Name: buildShadowVolume()
// Desc: Takes a mesh as input, and uses it to build a shadow volume. The
// technique used considers each triangle of the mesh, and adds it's
// edges to a temporary list. The edge list is maintained, such that
// only silohuette edges are kept. Finally, the silohuette edges are
// extruded to make the shadow volume vertex list.
//-----------------------------------------------------------------------------
void ShadowVolume::CheckDual(LPDIRECT3DDEVICE9 pd3dDevice)
{
D3DCAPS9 d3dCaps;
pd3dDevice->GetDeviceCaps( &d3dCaps );// D3DADAPTER_DEFAULT, D3DDEVTYPE_HAL,
if( ( d3dCaps.StencilCaps & D3DSTENCILCAPS_TWOSIDED ) != 0 )
g_bTwoSidedStencilsAvailable = true;
}
HRESULT ShadowVolume::buildShadowVolume( void* pVert,void* pInd,DWORD numFaces,DWORD numVerts,D3DXVECTOR3 vLight,D3DXMATRIX mat )
{
struct MeshVertex {
D3DXVECTOR3 p, n;
FLOAT tu,tv;};

MeshVertex *pVertices = (MeshVertex*)pVert;
WORD *pIndices = (WORD*)pInd;

DWORD dwNumFaces = numFaces;
DWORD dwNumVertices = numVerts;

WORD* pEdges = new WORD[dwNumFaces*6];
DWORD dwNumEdges = 0;

// For each face
for( DWORD i = 0; i < dwNumFaces; ++i )
{
WORD wFace0 = pIndices[3*i+0];
WORD wFace1 = pIndices[3*i+1];
WORD wFace2 = pIndices[3*i+2];
D3DXVECTOR3 v0 = pVertices[wFace0].p;
D3DXVECTOR3 v1 = pVertices[wFace1].p;
D3DXVECTOR3 v2 = pVertices[wFace2].p;

// Transform vertices or transform light?
D3DXVECTOR3 vCross1(v2-v1);
D3DXVECTOR3 vCross2(v1-v0);
D3DXVECTOR3 vNormal;
D3DXVec3Cross( &vNormal, &vCross1, &vCross2 );
D3DXVec3Normalize(&vNormal,&vNormal);

if( D3DXVec3Dot( &vNormal, &vLight ) >= 0.0f )
{
addEdge( pEdges, dwNumEdges, wFace0, wFace1 );
addEdge( pEdges, dwNumEdges, wFace1, wFace2 );
addEdge( pEdges, dwNumEdges, wFace2, wFace0 );
}
}

for( i = 0; i < dwNumEdges; ++i )
{
D3DXVECTOR3 v1 = pVertices[pEdges[2*i+0]].p;
D3DXVECTOR3 v2 = pVertices[pEdges[2*i+1]].p;
D3DXVECTOR3 v3 = v1 - vLight*20;
D3DXVECTOR3 v4 = v2 - vLight*20;

if (m_dwNumVertices<100000-7){
m_pVertices[m_dwNumVertices++] = v1;
m_pVertices[m_dwNumVertices++] = v2;
m_pVertices[m_dwNumVertices++] = v3;
m_pVertices[m_dwNumVertices++] = v2;
m_pVertices[m_dwNumVertices++] = v4;
m_pVertices[m_dwNumVertices++] = v3;
}
}

// Delete the temporary edge list
delete[] pEdges;

return S_OK;
}

//-----------------------------------------------------------------------------
// Name: addEdge()
// Desc: Adds an edge to a list of silohuette edges of a shadow volume.
//-----------------------------------------------------------------------------
void ShadowVolume::addEdge( WORD* pEdges, DWORD& dwNumEdges, WORD v0, WORD v1 )
{
//// Remove interior edges (which appear in the list twice)
//for( DWORD i = 0; i < dwNumEdges; ++i )
//{
// if( ( pEdges[2*i+0] == v0 && pEdges[2*i+1] == v1 ) ||
// ( pEdges[2*i+0] == v1 && pEdges[2*i+1] == v0 ) )
// {
// if( dwNumEdges > 1 )
// {
// pEdges[2*i+0] = pEdges[2*(dwNumEdges-1)+0];
// pEdges[2*i+1] = pEdges[2*(dwNumEdges-1)+1];
// }

// --dwNumEdges;
// return;
// }
//}

pEdges[2*dwNumEdges+0] = v0;
pEdges[2*dwNumEdges+1] = v1;
dwNumEdges++;
}




void ShadowVolume::renderShadowToStencilBuffer( LPDIRECT3DDEVICE9 g_pd3dDevice, D3DXMATRIX m_mat )
{


// Disable z-buffer writes (note: z-testing still occurs), and enable the
// stencil-buffer
DWORD shade,cull,zwrite,stencil,alpha;
g_pd3dDevice->GetRenderState( D3DRS_SHADEMODE, &shade );
g_pd3dDevice->GetRenderState( D3DRS_CULLMODE, &cull );
g_pd3dDevice->GetRenderState( D3DRS_ZWRITEENABLE, &zwrite );
g_pd3dDevice->GetRenderState( D3DRS_STENCILENABLE, &stencil );
g_pd3dDevice->GetRenderState( D3DRS_ALPHABLENDENABLE, &alpha );

g_pd3dDevice->SetRenderState( D3DRS_ZWRITEENABLE, FALSE );
g_pd3dDevice->SetRenderState( D3DRS_STENCILENABLE, TRUE );

// Dont bother with interpolating color
g_pd3dDevice->SetRenderState( D3DRS_SHADEMODE, D3DSHADE_FLAT );

// Set up stencil compare fuction, reference value, and masks.
// Stencil test passes if ((ref & mask) cmpfn (stencil & mask)) is true.
// Note: since we set up the stencil-test to always pass, the STENCILFAIL
// renderstate is really not needed.
g_pd3dDevice->SetRenderState( D3DRS_STENCILFUNC, D3DCMP_ALWAYS );
g_pd3dDevice->SetRenderState( D3DRS_STENCILZFAIL, D3DSTENCILOP_KEEP );
g_pd3dDevice->SetRenderState( D3DRS_STENCILFAIL, D3DSTENCILOP_KEEP );

// If z-test passes, inc/decrement stencil buffer value
g_pd3dDevice->SetRenderState( D3DRS_STENCILREF, 0x1 );
g_pd3dDevice->SetRenderState( D3DRS_STENCILMASK, 0xffffffff );
g_pd3dDevice->SetRenderState( D3DRS_STENCILWRITEMASK, 0xffffffff );
g_pd3dDevice->SetRenderState( D3DRS_STENCILPASS, D3DSTENCILOP_INCR );

// Make sure that no pixels get drawn to the frame buffer
g_pd3dDevice->SetRenderState( D3DRS_ALPHABLENDENABLE, TRUE );
g_pd3dDevice->SetRenderState( D3DRS_SRCBLEND, D3DBLEND_ZERO );
g_pd3dDevice->SetRenderState( D3DRS_DESTBLEND, D3DBLEND_ONE );

if( g_bTwoSidedStencilsAvailable == true )
{
// With 2-sided stencil, we can avoid rendering twice:
g_pd3dDevice->SetRenderState( D3DRS_TWOSIDEDSTENCILMODE, TRUE );
g_pd3dDevice->SetRenderState( D3DRS_CCW_STENCILFUNC, D3DCMP_ALWAYS );
g_pd3dDevice->SetRenderState( D3DRS_CCW_STENCILZFAIL, D3DSTENCILOP_KEEP );
g_pd3dDevice->SetRenderState( D3DRS_CCW_STENCILFAIL, D3DSTENCILOP_KEEP );
g_pd3dDevice->SetRenderState( D3DRS_CCW_STENCILPASS, D3DSTENCILOP_DECR );

g_pd3dDevice->SetRenderState( D3DRS_CULLMODE, D3DCULL_NONE );

// Draw both sides of shadow volume in stencil/z only
g_pd3dDevice->SetTransform( D3DTS_WORLD, &m_mat );
render( g_pd3dDevice );

g_pd3dDevice->SetRenderState( D3DRS_TWOSIDEDSTENCILMODE, FALSE );
}
else
{
// Draw front-side of shadow volume in stencil/z only
g_pd3dDevice->SetTransform( D3DTS_WORLD, &m_mat );
render( g_pd3dDevice );

// Now reverse cull order so back sides of shadow volume are written.
g_pd3dDevice->SetRenderState( D3DRS_CULLMODE, D3DCULL_CW );

// Decrement stencil buffer value
g_pd3dDevice->SetRenderState( D3DRS_STENCILPASS, D3DSTENCILOP_DECR );

// Draw back-side of shadow volume in stencil/z only
g_pd3dDevice->SetTransform( D3DTS_WORLD, &m_mat );
render( g_pd3dDevice );
}

// Restore render states
g_pd3dDevice->SetRenderState( D3DRS_SHADEMODE, shade );
g_pd3dDevice->SetRenderState( D3DRS_CULLMODE, cull );
g_pd3dDevice->SetRenderState( D3DRS_ZWRITEENABLE, zwrite );
g_pd3dDevice->SetRenderState( D3DRS_STENCILENABLE, stencil );
g_pd3dDevice->SetRenderState( D3DRS_ALPHABLENDENABLE, alpha );
}



void ShadowVolume::renderShadowToScene( LPDIRECT3DDEVICE9 g_pd3dDevice )
{
// Set render states
g_pd3dDevice->SetRenderState( D3DRS_ZENABLE, FALSE );
g_pd3dDevice->SetRenderState( D3DRS_STENCILENABLE, TRUE );
g_pd3dDevice->SetRenderState( D3DRS_ALPHABLENDENABLE, TRUE );
g_pd3dDevice->SetRenderState( D3DRS_SRCBLEND, D3DBLEND_SRCALPHA );
g_pd3dDevice->SetRenderState( D3DRS_DESTBLEND, D3DBLEND_INVSRCALPHA );

g_pd3dDevice->SetTextureStageState( 0, D3DTSS_COLORARG1, D3DTA_TEXTURE );
g_pd3dDevice->SetTextureStageState( 0, D3DTSS_COLORARG2, D3DTA_DIFFUSE );
g_pd3dDevice->SetTextureStageState( 0, D3DTSS_COLOROP, D3DTOP_MODULATE );
g_pd3dDevice->SetTextureStageState( 0, D3DTSS_ALPHAARG1, D3DTA_TEXTURE );
g_pd3dDevice->SetTextureStageState( 0, D3DTSS_ALPHAARG2, D3DTA_DIFFUSE );
g_pd3dDevice->SetTextureStageState( 0, D3DTSS_ALPHAOP, D3DTOP_MODULATE );

// Only write where stencil value >= 1 (count indicates # of shadows that
// overlap that pixel)
g_pd3dDevice->SetRenderState( D3DRS_STENCILREF, 0x1 );
g_pd3dDevice->SetRenderState( D3DRS_STENCILFUNC, D3DCMP_LESSEQUAL );
g_pd3dDevice->SetRenderState( D3DRS_STENCILPASS, D3DSTENCILOP_KEEP );

// Draw a big, gray square
g_pd3dDevice->SetFVF( ShadowVertex::FVF_Flags );
g_pd3dDevice->SetStreamSource( 0, m_pBigSquareVB, 0, sizeof(ShadowVertex) );
g_pd3dDevice->DrawPrimitive( D3DPT_TRIANGLESTRIP, 0, 2 );

// Restore render states
g_pd3dDevice->SetRenderState( D3DRS_ZENABLE, TRUE );
g_pd3dDevice->SetRenderState( D3DRS_STENCILENABLE, FALSE );
g_pd3dDevice->SetRenderState( D3DRS_ALPHABLENDENABLE, FALSE );
}

Share this post


Link to post
Share on other sites
wusher, I am going through the plane, when I render it in wireframe I can see it go through my ground and down below that pretty far.

Jeff.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
reverse the vertex winding of the extruded faces based on which face of the edge is back facing...

since edges are shared between triangles, that means each edges has 2 correct windings, one for each triangle.

i hope this is making some sense.

Share this post


Link to post
Share on other sites
Uhm, not really sure. Are you saying when I add the vertices like this:

m_pVertices[m_dwNumVertices++] = v1;
m_pVertices[m_dwNumVertices++] = v2;
m_pVertices[m_dwNumVertices++] = v3;
m_pVertices[m_dwNumVertices++] = v2;
m_pVertices[m_dwNumVertices++] = v4;
m_pVertices[m_dwNumVertices++] = v3;

Its not always the case and that some should be reversed?? If so, how do I check that? Or am I totally not getting what your saying.

Thanks for helping, this is insane. Also, if I leave in the function to remove vertices that appear twice in the list you actually see the entire wireframe shadow volume and it looks all messed up. I leave that out and you get what you see in the screenshot. Just another note.

Thanks
Jeff.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
okay,

there seems to be a lot wrong with how you're doing the whole
buildShadowVolume thing... you appear to be adding 3 edges for each triangle facing the light... it really should be closer to 1.

it should go something like:

build an edge list, each edge stores a pointer to 2 verts and 2 triangles.

for every edge
if egde->tri1 facing is not equal to edge->tri2 facing
add the 2 verts of this edge and the 2 verts of this edge
extruded to infinity.

Share this post


Link to post
Share on other sites
Hi,

It looks very much like a problem concerning polygon list.

I noticed that on certain models my shadow volume had holes of that type.

The problem turned out to be that the shadow volume algorithm had to operate on a triangle list - the mesh I was using contained triangle fans and strips.

I think that you should try to use separate models for the shadow and the model. If you try to ensure that the model used for the shadow contains no triangle strips or fans then it should work.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!