7 replies to this topic

[L. Spiro]

The DirectX 11 side of my engine is 95% there. I only need to fix 2 problems. One of which is this shadow problem. Although I have a lot of experience with graphics programming, I am in a bit of a mess of a mental state right now and I can’t easily recognize what type of thing could cause this type of shadow error. PIX won’t show me the an image of the depth texture, which is another question I will be asking.

Here are the results:


Everything that should be shadowed is. But with some additional shadows where there should not be.
The overall pipeline is exactly the same as in DirectX 9, OpenGL 3.2, and OpenGL ES 2, all of which have perfectly fine shadows, so I have a feeling this is some kind of DirectX 11 semantic issue.

Here is the shader I am using. This is machine-generated so I added back some whitespace etc. to make it readable by humans.

SamplerState lsg_SamplerShadow : register( s15 ) {
	Filter = MIN_MAG_LINEAR_MIP_POINT;
	AddressU = CLAMP;
	AddressV = CLAMP;
};
float shadow2dDepth( Texture2D _tTexture, float2 _vCoord ) {
	return _tTexture.Sample( lsg_SamplerShadow, _vCoord ).x;
}
Texture2D g_sShadowTex : register( t15 );


cbuffer cb0 : register( b0 ) {
	vector<float, 4> g_vDiffuseMaterial : packoffset( c0.x );
	vector<float, 4> g_vSpecularMaterial : packoffset( c1.x );
	float g_fPower : packoffset( c2.x );
};
cbuffer cb1 : register( b1 ) {
	vector<float, 4> g_vLightVectors[8] : packoffset( c0.x );
	vector<float, 4> g_vLightAmbients[8] : packoffset( c8.x );
	vector<float, 4> g_vLightDiffuses[8] : packoffset( c16.x );
	vector<float, 4> g_vLightSpeculars[8] : packoffset( c24.x );
};
cbuffer cb3 : register( b3 ) {
	int g_iTotalDirLights : packoffset( c0.x );
	matrix<float, 4, 4> g_mShadowMapMatrix : packoffset( c1.x );
};

struct LSE_COLOR_PAIR {
	vector<float, 4> cDiffuse;
	vector<float, 4> cSpecular;
};

LSE_COLOR_PAIR GetDirLightColors( in vector<float, 3> _vNormalInViewSpace, in vector<float, 4> _vViewVector, in int _iIndex ) {
	LSE_COLOR_PAIR cpRet;
	vector<float, 3> vLightDir = g_vLightVectors[_iIndex].xyz;
	float fNormalDotLight = dot( vLightDir, _vNormalInViewSpace );
	cpRet.cDiffuse = g_vLightAmbients[_iIndex];
	cpRet.cSpecular = vector<float, 4>( 0.0, 0.0, 0.0, 0.0 );
	if( fNormalDotLight> 0.0 ) {
		cpRet.cDiffuse += fNormalDotLight * g_vLightDiffuses[_iIndex];
		cpRet.cSpecular = pow( max( 0.0, dot( normalize( vLightDir + _vViewVector.xyz ), _vNormalInViewSpace ) ), g_fPower ) * g_vLightSpeculars[_iIndex];
	}
	return cpRet;
}

float ShadowLookUp( in vector<float, 2> _vPos, in float _zDivide, in vector<float, 4> _vShadowCoord ) {
	float fShadow = 1.0;
	if( _vShadowCoord.w > 0.0 && _vShadowCoord.x >= 0.0 && _vShadowCoord.x <= 1.0 && _vShadowCoord.y >= 0.0 && _vShadowCoord.y <= 1.0 ) {
		float fDepth = shadow2dDepth( g_sShadowTex, _vPos );
		if( fDepth != 1.0 ) {
			fShadow = fDepth < _zDivide ? 0.0 : 1.0;
		}
	}
	return fShadow;
}
float StandardShadowMap( in vector<float, 4> _vShadowCoord ) {
	vector<float, 4> vShadowCoordWDivide = _vShadowCoord / _vShadowCoord.w;
	vShadowCoordWDivide.z -= 0.00062500000000000001 * 0.5;
	return ShadowLookUp( vShadowCoordWDivide.xy, vShadowCoordWDivide.z, _vShadowCoord );
}

void Main( in vector<float, 3> _vInNormal : NORMAL0, in vector<float, 2> _vIn2dTex0 : TEXCOORD2, in vector<float, 4> _vInPos : SV_POSITION0, in vector<float, 4> _vInEyePos : TEXCOORD1, out vector<float, 4> _vOutColor : SV_Target0 ) {
	vector<float, 4> vShadowCoord = mul( g_mShadowMapMatrix, _vInEyePos );
	float fShadow = StandardShadowMap( vShadowCoord );
	vector<float, 4> vColorTemp;
	vector<float, 3> vNormalizedNormal = normalize( _vInNormal );
	vector<float, 4> vViewPosToEye = -normalize( _vInEyePos );
	LSE_COLOR_PAIR cpLightColors = {
		vector<float, 4>( 0.0, 0.0, 0.0, 0.0 ),
		vector<float, 4>( 0.0, 0.0, 0.0, 0.0 )
	};
	for( int I = 0; I < g_iTotalDirLights; I++ ) {
		LSE_COLOR_PAIR cpThis = GetDirLightColors( vNormalizedNormal, vViewPosToEye, I );
		cpLightColors.cDiffuse += cpThis.cDiffuse;
		cpLightColors.cSpecular += cpThis.cSpecular;
	}
	_vOutColor = g_vDiffuseMaterial;
	_vOutColor.xyz = (_vOutColor.xyz * cpLightColors.cDiffuse.xyz) + (g_vSpecularMaterial * cpLightColors.cSpecular).xyz;
	_vOutColor.xyz *= fShadow;
	_vOutColor = max( _vOutColor, vector<float, 4>( 0.0, 0.0, 0.0, 0.0 ) );
}

Since the overall shader is exactly the same in DirectX 9, I suspect it might have to do with the samplers or the way in which I read from the depth texture.
Here is how I am making the depth texture.

/**
	 * Creates a Direct3D 11 depth texture.
	 *
	 * \return Returns true if the creation of the texture succeeds.  False indicates a resource error.
	 */
	LSBOOL LSE_CALL CDirectX11FloatTexture::CreateApiTexture() {
		ResetApi();
		D3D11_TEXTURE2D_DESC tdDec = { 0 };
		tdDec.Width = m_ui32Width;
		tdDec.Height = m_ui32Height;
		tdDec.MipLevels = 1;
		tdDec.ArraySize = 1;
		tdDec.Format = DXGI_FORMAT_R32_TYPELESS;
		tdDec.SampleDesc.Count = 1;
		tdDec.Usage = D3D11_USAGE_DEFAULT;
		tdDec.BindFlags = D3D11_BIND_DEPTH_STENCIL | D3D11_BIND_SHADER_RESOURCE;
		tdDec.CPUAccessFlags = D3D11_USAGE_DEFAULT;
		if ( FAILED( CDirectX11::GetDirectX11Device()->CreateTexture2D( &tdDec, NULL, &m_pt2tTexture ) ) ) {
			ResetApi();
			return false;
		}
	  
		// Create the description of the render target view.
		D3D11_DEPTH_STENCIL_VIEW_DESC dsvDepthStencilView = { DXGI_FORMAT_D32_FLOAT };
		dsvDepthStencilView.ViewDimension = D3D11_DSV_DIMENSION_TEXTURE2D;
		dsvDepthStencilView.Texture2D.MipSlice = 0;
		// Create the render target view.
		if ( FAILED( CDirectX11::GetDirectX11Device()->CreateDepthStencilView( m_pt2tTexture, &dsvDepthStencilView, &m_pdsvView ) ) ) {
			ResetApi();
			return false;
		}

		// Setup the description of the shader resource view.
		D3D11_SHADER_RESOURCE_VIEW_DESC srvdShaderResourceViewDesc = { DXGI_FORMAT_R32_FLOAT };
		srvdShaderResourceViewDesc.ViewDimension = D3D11_SRV_DIMENSION_TEXTURE2D;
		srvdShaderResourceViewDesc.Texture2D.MostDetailedMip = 0;
		srvdShaderResourceViewDesc.Texture2D.MipLevels = 1;
		// Create the shader resource view.
		if ( FAILED( CDirectX11::GetDirectX11Device()->CreateShaderResourceView( m_pt2tTexture, &srvdShaderResourceViewDesc, &m_psrvResourceView ) ) ) {
			ResetApi();
			return false;
		}

		// Create the sampler for this texture.
		D3D11_SAMPLER_DESC sdSamplerDesc;
		sdSamplerDesc.Filter = D3D11_FILTER_MIN_MAG_MIP_LINEAR;
		sdSamplerDesc.AddressU = D3D11_TEXTURE_ADDRESS_CLAMP;
		sdSamplerDesc.AddressV = D3D11_TEXTURE_ADDRESS_CLAMP;
		sdSamplerDesc.AddressW = D3D11_TEXTURE_ADDRESS_CLAMP;
		sdSamplerDesc.MipLODBias = 0.0f;
		sdSamplerDesc.MaxAnisotropy = 1;
		sdSamplerDesc.ComparisonFunc = D3D11_COMPARISON_ALWAYS;
		sdSamplerDesc.BorderColor[0] = 0.0f;
		sdSamplerDesc.BorderColor[1] = 0.0f;
		sdSamplerDesc.BorderColor[2] = 0.0f;
		sdSamplerDesc.BorderColor[3] = 0.0f;
		sdSamplerDesc.MinLOD = 0.0f;
		sdSamplerDesc.MaxLOD = 0.0f;
		if ( FAILED( CDirectX11::GetDirectX11Device()->CreateSamplerState( &sdSamplerDesc, &m_pssSampler ) ) ) {
			ResetApi();
			return false;
		}
		return true;
	}

The texture and sampler are being activated in registers t15 and s15 respectively.


#1: Is there anything obvious I am doing wrong? A bad mix of states or flags or-? PIX is not helping and debug DirectX prints no errors/warnings/infos.
#2: The sampler in the shader gets overridden by m_pssSampler when it is activated, right?
#3: Is there any way to get PIX to show me this texture? It would help if I could see the texture.


L. Spiro

Edited by L. Spiro, 20 July 2012 - 10:01 PM.

[eppo]

For some reason (fDepth < _zDivide) always returns zero.

The floor object isn't affected by its own shadow map values, because it doesn't cast a shadow? That would mean for those parts of the map, the sampler would simply return the Clear()-value. Best to inspect the shadow map in PIX.

I've also had difficulties viewing a FLOAT_32 texture in PIX ('surface cannot be shown' etc.). Using a DXGI_FORMAT_R24G8_TYPELESS/R24_UNORM/D24_UNORM combo should work better.

Also, assigning values to a SamplerState object only has meaning when you use the effects framework. In that case the framework will internally create an ID3D11SamplerState object based of those values, but now you're forced to bind samplers yourself (or the debug layer will give you a warning).

Edited by eppo, 21 July 2012 - 04:59 AM.

[L. Spiro]

For some reason (fDepth < _zDivide) always returns zero.

It doesn’t always return 0 or else all of the pig would be shadowed.
I would also like to note that there is no shadow just along the border where lighting normally ends and shadows begin. You can see it if you look closely.
Then there is shadow above and below it. Above is wrong.

The floor object isn't affected by its own shadow map values, because it doesn't cast a shadow?

Correct. Only the pig is set to cast shadows.


L. Spiro

[mind in a box]

Is the shadowmap rendered with an orthogonal matrix? I had to set my camera very high over my scene to not get similar results like you.
But honestly don't think that this is only a D3D11-thing.

[mark ds]

Would I be right in thinking that the cows feet are at zero? (which ever axis they may be on).

It looks like a negation of a certain axis to me, because of the way the 'pink' area points back to the floor plane.

[MJP]

For whatever reason PIX won't let you view R32_TYPELESS depth textures. It works fine with R24G8_TYPELESS.

[L. Spiro]

Is the shadowmap rendered with an orthogonal matrix?

Yes. It is a directional light.

Would I be right in thinking that the cows feet are at zero? (which ever axis they may be on).

It looks like a negation of a certain axis to me, because of the way the 'pink' area points back to the floor plane.

That is 0 with y = up. But I don’t see negated axes.

For whatever reason PIX won't let you view R32_TYPELESS depth textures. It works fine with R24G8_TYPELESS.

I will try it. It would help a lot to know if the texture itself is wrong or if my usage of the texture is wrong.


L. Spiro

[xoofx]

If you have a NVidia graphics card, I would defintely try to debug it with NSight, as It is much easier to inspect the state of the whole pipeline and you can see the depth buffer and inspect depth values as well.