Shadow Mapping Woes

Started by
26 comments, last by L. Spiro 12 years, 5 months ago
Shadow mapping works fine in OpenGL. Trying to catch up my DirectX side but nothing is ever rendered to the depth texture.

I have followed the sample in the Sample Browser to a tee as far as I can tell, but I must have missed something. I am fairly sure it has only to do with the depth texture itself, since I can confirm that the framework for shadow mapping is fine (setting of the light’s projection matrix, gathering objects the light can see, etc.) because it works in OpenGL, and obviously all of that code is the same between both builds.

This is the correct result via OpenGL:
shadowmaps.png

In Direct3D 9, it is all just white, indicating either I didn’t draw anything to it, or I am not reading from it properly when I try to show it on my screen (less likely).
(By the way I do take measures to ensure it is not bound as both a texture and a render target.)

So, here is my code, which replicates the sample exactly as far as I can see.


Create the texture:


/**
* Creates a Direct3D 9 depth texture.
*
* \return Returns true if the creation of the texture succeeds. False indicates a resource error.
*/
LSBOOL LSE_CALL CDirectX9FloatTexture::CreateApiTexture() {
CDirectX9::SafeRelease( m_pd3tTexture );
CDirectX9::SafeRelease( m_pd3dsDepthSurface );
if ( FAILED( CDirectX9::GetDirectX9Device()->CreateTexture( m_ui32Width, m_ui32Height,
1UL, D3DUSAGE_RENDERTARGET,
D3DFMT_R32F,
D3DPOOL_DEFAULT,
&m_pd3tTexture,
NULL ) ) ) { return false; }

// Create a depth/stencil buffer of the same size.
if ( FAILED( CDirectX9::GetDirectX9Device()->CreateDepthStencilSurface( m_ui32Width, m_ui32Height,
CDirectX9::GetPresentParms().AutoDepthStencilFormat,
D3DMULTISAMPLE_NONE,
0,
TRUE,
&m_pd3dsDepthSurface,
NULL ) ) ) {
CDirectX9::SafeRelease( m_pd3tTexture );
return false;
}

return true;
}




Set it as a render target (makes a back-up of the existing render target).


/**
* Activate this texture as a render target.
*
* \return Returns true if the texture is ready to be used as a render target.
*/
LSBOOL LSE_CALL CDirectX9FloatTexture::SetAsRenderTarget() const {
if ( !m_pd3tTexture ) { return false; }

// Back up the current render targets.
if ( !m_pd3d9sPrevSurface.GetRenderTarget( true, 0UL ) ) { return false; }
if ( !m_pd3d9sPrevDepthSurface.GetRenderTarget( false, 0UL ) ) { return false; } // false = get the depth surface.

// Set the float buffer.
IDirect3DSurface9 * pd3dsSurface;
if ( FAILED( m_pd3tTexture->GetSurfaceLevel( 0, &pd3dsSurface ) ) ) { return false; }
if ( FAILED( CDirectX9::GetDirectX9Device()->SetRenderTarget( 0, pd3dsSurface ) ) ) {
CDirectX9::SafeRelease( pd3dsSurface );
return false;
}
CDirectX9::SafeRelease( pd3dsSurface );

// Set the depth/stencil buffer.
if ( FAILED( CDirectX9::GetDirectX9Device()->SetDepthStencilSurface( m_pd3dsDepthSurface ) ) ) {
// Put the old render target back too.
m_pd3d9sPrevSurface.ReApply();
return false;
}


CFnd::SaveViewPort();
CFnd::SetViewport( 0, 0, GetWidth(), GetHeight() );
return true;
}




Render render render.
Then stop being a render target.



/**
* End its term as a render target.
*/
LSVOID LSE_CALL CDirectX9FloatTexture::EndAsRenderTarget() const {
// Puts the old surfaces back. One render target and one depth/stencil.
m_pd3d9sPrevSurface.ReApply();
m_pd3d9sPrevDepthSurface.ReApply();

CFnd::RestoreViewPort();
}




Apparently the error is in there, but perhaps I fail at displaying it on the screen, so here is how it is shown on the screen.
Activate it into a texture slot.



/**
* Activate this texture in a given slot.
*
* \param _ui32Slot Slot in which to place this texture.
* \return Returns true if the texture is activated successfully.
*/
LSBOOL LSE_CALL CDirectX9FloatTexture::Activate( LSUINT32 _ui32Slot ) {
if ( !m_pd3tTexture ) { return false; }
CCriticalSection::CLocker lLock( m_csBaseCrit );
if ( m_ui32LastTextures[_ui32Slot] != m_ui32Id ) {
m_ui32LastTextures[_ui32Slot] = m_ui32Id;
if ( m_ui32LastSlot != ~0UL ) {
// If we were bound to another slot, unbind from there.
m_ui32LastTextures[m_ui32LastSlot] = 0UL;
if ( FAILED( CDirectX9::GetDirectX9Device()->SetTexture( m_ui32LastSlot, NULL ) ) ) { return false; }
}
m_ui32LastSlot = _ui32Slot;
if ( FAILED( CDirectX9::GetDirectX9Device()->SetTexture( _ui32Slot, m_pd3tTexture ) ) ) { return false; }
if ( FAILED( CDirectX9::GetDirectX9Device()->SetSamplerState( _ui32Slot, D3DSAMP_MAGFILTER, D3DTEXF_POINT ) ) ) { return false; }
if ( FAILED( CDirectX9::GetDirectX9Device()->SetSamplerState( _ui32Slot, D3DSAMP_MINFILTER, D3DTEXF_POINT ) ) ) { return false; }
if ( FAILED( CDirectX9::GetDirectX9Device()->SetSamplerState( _ui32Slot, D3DSAMP_MIPFILTER, D3DTEXF_NONE ) ) ) { return false; }
if ( FAILED( CDirectX9::GetDirectX9Device()->SetSamplerState( _ui32Slot, D3DSAMP_MAXANISOTROPY, 1UL ) ) ) { return false; }


if ( FAILED( CDirectX9::GetDirectX9Device()->SetSamplerState( _ui32Slot, D3DSAMP_ADDRESSU, D3DTADDRESS_CLAMP ) ) ) { return false; }
if ( FAILED( CDirectX9::GetDirectX9Device()->SetSamplerState( _ui32Slot, D3DSAMP_ADDRESSV, D3DTADDRESS_CLAMP ) ) ) { return false; }
}
return true;
}




And it can’t be bound and a render target, so I unbind it as follows.


/**
* Remove this texture from the last slot in which it was placed.
*/
LSVOID LSE_CALL CDirectX9FloatTexture::DeActivate() {
if ( m_ui32LastSlot != ~0UL ) {
CDirectX9::GetDirectX9Device()->SetTexture( m_ui32LastSlot, NULL );
m_ui32LastTextures[m_ui32LastSlot] = 0UL;
m_ui32LastSlot = ~0UL;
}
}




I render to to screen by accessing its .z and copying it into the .x and .y, and setting .w to 1.0f.


Did I miss something?


L. Spiro

I restore Nintendo 64 video-game OST’s into HD! https://www.youtube.com/channel/UCCtX_wedtZ5BoyQBXEhnVZw/playlists?view=1&sort=lad&flow=grid

Advertisement
You might not be doing anything wrong.
Because the 'z' is not linear the values you see when you render it are approaching 1, or white.
If you get really close to something, you should see some more shades of gray.

Can you save the data out and verify the values are not all 1?
The view is sized so that the farthest object will be virtually white and the nearest virtually black.
It is the exact same view matrix as the one used in OpenGL, so the result will look like the image posted above.
I have also confirmed I am feeding in the exact same matrices in both implementations.

I am definitely doing something wrong somewhere.
I certainly can’t see it here though.

What other kinds of subtle differences are there between shadow mapping in OpenGL and in Direct3D 9?
Shadow mapping is fully functional in my OpenGL side. Not just printing the depth texture to the screen but also shadows on the actual objects themselves.

The shaders I use for both are logically identical. I write only one shader and my engine converts it to GLSL or HLSL, so the logic in both are always identical.

But for now that isn’t the concern. I can’t get anything onto the depth texture at all.
I used debug-mode DirectX and it did not print anything related to the depth texture.


L. Spiro

I restore Nintendo 64 video-game OST’s into HD! https://www.youtube.com/channel/UCCtX_wedtZ5BoyQBXEhnVZw/playlists?view=1&sort=lad&flow=grid

Just to confirm .. does your shader output depth manually as the output color in both GLSL & HLSL? The Direct3D code shown in your post requires that to work properly.

OpenGL shadow mapping typically uses a GL_DEPTH_COMPONENT format texture (bound as the depth attachment while rendering shadow casters), in which case this would not be necessary, as the hardware depth can be directly sampled from that texture.

To utilize hardware shadow mapping on Direct3D, one would create a depth stencil texture in a format such as D3DFMT_D24X8, bind that as the depth stencil while rendering the shadow casters, then sample it as a texture. When doing this, the color rendertarget (and what the pixel shader writes to it) is irrelevant, the Direct3D API just requires that some equally sized rendertarget is bound. It is somewhat of a hack but supported by all major GPU vendors by now. See http://aras-p.info/t...D9GPUHacks.html for more info. That page also includes the trick of creating a "NULL" format dummy rendertarget (if supported, slightly older GPUs do not) for depth-only rendering so that it doesn't consume video memory at all.
I did not notice that.
My shaders now look like this:


matrix<float,4,4> g_mModelViewProjMatrix;


void Main( in vector<float,3> _vInPos : POSITION0,
out vector<float,4> _vOutPos : POSITION0,
out vector<float,4> _vOutDepth : TEXCOORD0 ) {
_vOutPos = mul( g_mModelViewProjMatrix, vector<float,4>( _vInPos, 1.0 ) );

_vOutDepth.xy = _vOutPos.zw;
}



void Main( in vector<float,4> _vInPos : POSITION0,
in vector<float,4> _vInDepth : TEXCOORD0,
out vector<float,4> _vOutColor : COLOR0 ) {
_vOutColor = (_vInDepth.x / _vInDepth.y);
}



But the result is the same.


L. Spiro

I restore Nintendo 64 video-game OST’s into HD! https://www.youtube.com/channel/UCCtX_wedtZ5BoyQBXEhnVZw/playlists?view=1&sort=lad&flow=grid

Do you clear the render target / depth buffer when they are set?
Yes. To 1.0f.


L. Spiro

I restore Nintendo 64 video-game OST’s into HD! https://www.youtube.com/channel/UCCtX_wedtZ5BoyQBXEhnVZw/playlists?view=1&sort=lad&flow=grid

If you try debugging with PIX, what does it show for the shadowmap texture? (you can click a pixel in the texture and it shows the pixel's history, telling if it was for example rejected due to depth test.) Also, if you modify your shader to output 0.0f, do you then see the shadowcasters in the texture?
Something about that picture up there strikes me as odd.
That's a depth buffer right?
Why is the point the human arm connects to the body a different shade? They should be at the same depth.
Same with the dino legs, and the bullet belt / holster. Seems, odd, no?

If you try debugging with PIX, what does it show for the shadowmap texture? (you can click a pixel in the texture and it shows the pixel's history, telling if it was for example rejected due to depth test.) Also, if you modify your shader to output 0.0f, do you then see the shadowcasters in the texture?

I don’t know a good way to debug it because when I click a pixel from the back buffer it shows me what happened on the back buffer, but not on my special buffer.
I will have to do a lot of rearranging to show only the back buffer.
To be noted is that the other tutorial samples I have downloaded also do not work. Only the Microsoft sample does, but I can’t see why it does and mine does not.

Also I output color [0, 0, 0, 0] but still see a plain-white square.



Something about that picture up there strikes me as odd.
That's a depth buffer right?
Why is the point the human arm connects to the body a different shade? They should be at the same depth.
Same with the dino legs, and the bullet belt / holster. Seems, odd, no?

It may look strange but it is correct. It produces beautiful results in OpenGL.

bmwx64.png


hunter0.png


spitfire0.png



L. Spiro

I restore Nintendo 64 video-game OST’s into HD! https://www.youtube.com/channel/UCCtX_wedtZ5BoyQBXEhnVZw/playlists?view=1&sort=lad&flow=grid

This topic is closed to new replies.

Advertisement