Sign in to follow this  
riuthamus

Cascaded Shadow Map Issue

Recommended Posts

We have been trying to get cascaded shadow maps working from MJP's sample but I can't seem to get it working right. All I get is a diagonal area of shadow across the ground regardless of which direction I look or where the light is. From my debugging of the shader, it seems like the problem is that the shader that builds the occlusion map is sampling the cascade map from the wrong point. I've gone over the code several times and can't see what would be causing the issue.

 

gallery_1_8_26705.png

gallery_1_8_189040.png


This is the part of the shader that calculates the texture coordinate which seems to be wrong:
 

// Reconstruct
view-space position from the depth buffer
float pixelDepth =
DepthMap.Sample(DepthMapSampler, input.TexCoord).r;
float4 position =
float4(pixelDepth * input.FrustumCornerVS, 1.0f);

// Determine the depth
of the pixel with respect to the light
float4x4 inverseLVP = mul(InverseView,
lightViewProjection);
float4 positionLight = mul(position,
inverseLVP);

float lightDepth = (positionLight.z /
positionLight.w);

// Transform from light space to shadow map texture
space.
float2 shadowTexCoord = 0.5 * positionLight.xy / positionLight.w +
float2(0.5f, 0.5f);
shadowTexCoord.x = shadowTexCoord.x / SPLITS +
offset;
shadowTexCoord.y = 1.0f - shadowTexCoord.y;

// Offset the
coordinate by half a texel so we sample it correctly
shadowTexCoord += (0.5f
/ ShadowMapSize);

 

Any help with this would be greatly appreciated, thanks in advance.

Edited by riuthamus

Share this post


Link to post
Share on other sites

My guess would be that your not creating your orthogonal projection matrix for the shadow map wide enough to contain the whole scene. Try widening that projection matrix and see if the problem is resolved.

Edited by Nyssa

Share this post


Link to post
Share on other sites
Why would that cause the sampling position to be wrong? And how would I just make it bigger? The code for the frustum building is also from MJP's sample, so there shouldn't be any fundamental problems with it unless I ported it wrong.

Share this post


Link to post
Share on other sites

Without seeing your code It's hard to say exactly where the issue is. The shader code you posted looks alright. From the images you have posted I've seen that issue before when the shadow frustum hasn't been built correctly and/or the frustum is missing a transform into light space. Maybe also look over the matrices you're passing into the shader to make sure they are correct.

Edited by Nyssa

Share this post


Link to post
Share on other sites
Well, here's the code for the frustum creation. It's possible the issue is from convention differences between XNA (what the sample uses) and SharpDX (which is what we're using), but I wouldn't really know
public static void Draw()
{
	LightDirection = Environment.SkyDome.SunLight.LightDirection;
	Engine.Context.OutputMerger.SetTargets(depthStencil.DepthStencilView, depthMap);
	Engine.Context.ClearRenderTargetView(depthMap, Color.White);

	// Get corners of the main camera's bounding frustum
	var viewMatrix = World.Camera.View;
	frustumCornersWS = World.Camera.BoundingFrustum.GetCorners();

	//frustumCornersVS = Vector3.Transform(frustumCornersWS, ref viewMatrix);
	var vector = new Vector4[8];
	Vector3.Transform(frustumCornersWS, ref viewMatrix, vector);
	for(var i = 0; i < 8; i++)
	{
		frustumCornersVS[i] = new Vector3(vector[i].X, vector[i].Y, vector[i].Z);
	}
	for(var i = 0; i < 4; i++)
	{
		farFrustumCornersVS[i] = frustumCornersVS[i + 4];
	}
	// Calculate the cascade splits.  We calculate these so that each successive
	// split is larger than the previous, giving the closest split the most amount
	// of shadow detail.  
	const float n = Splits;
	const float near = 1;
	var far = World.Camera.FarPlaneDistance;
	splitDepths[0] = near;
	splitDepths[Splits] = far;
	const float splitConstant = 0.95f; //0.95f
	for(var i = 1; i < splitDepths.Length - 1; i++)
	{
		splitDepths[i] = splitConstant * near * (float)Math.Pow(far / near, i / n) + (1.0f - splitConstant) * ((near + (i / n)) * (far - near));
	}
	// Render our scene geometry to each split of the cascade
	for(var i = 0; i < Splits; i++)
	{
		var minZ = splitDepths[i];
		var maxZ = splitDepths[i + 1];

		lightCameras[i] = CalculateFrustum(minZ, maxZ);

		RenderDepthMap(i);
	}

	Engine.Context.Rasterizer.SetViewports(Engine.Viewport);
	RenderShadowMap();
}

private static OrthographicCamera CalculateFrustum(float minZ, float maxZ)
{
	// Shorten the view frustum according to the shadow view distance
	var cameraMatrix = World.Camera.World;

	for(var i = 0; i < 4; i++)
		splitFrustumCornersVS[i] = frustumCornersVS[i + 4] * (minZ / World.Camera.FarPlaneDistance);

	for(var i = 4; i < 8; i++)
		splitFrustumCornersVS[i] = frustumCornersVS[i] * (maxZ / World.Camera.FarPlaneDistance);

	var vector = new Vector4[8];
	Vector3.Transform(splitFrustumCornersVS, ref cameraMatrix, vector);

	for(var i = 0; i < 8; i++)
	{
		frustumCornersWS[i] = new Vector3(vector[i].X, vector[i].Y, vector[i].Z);
	}

	// Position the shadow-caster camera so that it's looking at the centroid,
	// and backed up in the direction of the sunlight
	var viewMatrix = Matrix.LookAtRH(Vector3.Zero - (LightDirection*100), Vector3.Zero, new Vector3(0, 1, 0));

	// Determine the position of the frustum corners in light space
	Vector3.Transform(frustumCornersWS, ref viewMatrix, vector);

	for(var i = 0; i < 8; i++)
	{
		frustumCornersLS[i] = new Vector3(vector[i].X, vector[i].Y, vector[i].Z);
	}

	// Calculate an orthographic projection by sizing a bounding box 
	// to the frustum coordinates in light space
	var mins = frustumCornersLS[0];
	var maxes = frustumCornersLS[0];
	for(var i = 1; i < 8; i++)
	{
		mins = Vector3.Min(mins, frustumCornersLS[i]);
		maxes = Vector3.Max(maxes, frustumCornersLS[i]);
	}
	if(toggleJitter)
	{
		// We snap the camera to 1 pixel increments so that moving the camera does not cause the shadows to jitter.
		// This is a matter of integer dividing by the world space size of a texel
		var diagonalLength = (frustumCornersWS[0] - frustumCornersWS[6]).Length();
		diagonalLength += 2;    //Without this, the shadow map isn't big enough in the world.
		var worldsUnitsPerTexel = diagonalLength / (ShadowResolution * Splits);

		var vBorderOffset = (new Vector3(diagonalLength, diagonalLength, diagonalLength) - (maxes - mins)) * 0.5f;
		maxes += vBorderOffset;
		mins -= vBorderOffset;

		mins /= worldsUnitsPerTexel;
		mins.X = (float)Math.Floor(mins.X);
		mins.Y = (float)Math.Floor(mins.Y);
		mins.Z = (float)Math.Floor(mins.Z);
		mins *= worldsUnitsPerTexel;

		maxes /= worldsUnitsPerTexel;
		maxes.X = (float)Math.Floor(maxes.X);
		maxes.Y = (float)Math.Floor(maxes.Y);
		maxes.Z = (float)Math.Floor(maxes.Z);
		maxes *= worldsUnitsPerTexel;
	}

	var lightCamera = new OrthographicCamera(mins.X, maxes.X, mins.Y, maxes.Y, -maxes.Z - nearClipOffset, -mins.Z);

	lightCamera.SetViewMatrix(ref viewMatrix);

	return lightCamera;
}

Share this post


Link to post
Share on other sites

I haven't used SlimDX before so I can't comment on its usage. I did see a problem with how you are positioning the frustum. Specifically in the following line:

 

var viewMatrix = Matrix.LookAtRH(Vector3.Zero - (LightDirection*100), Vector3.Zero, new Vector3(0, 1, 0));

 

This code means the shadow frustum never moves with the camera. MJP's examples calculates the frustum center (instead of using Vector3.Zero) with the following code:

 

// Find the centroid
Vector3 frustumCentroid = new Vector3(0,0,0);
for (int i = 0; i < 8; i++)
	frustumCentroid += frustumCornersWS[i];
frustumCentroid /= 8;

 

There is one limitation with the example MJP gives as well that may be problematic for you. Only the viewable objects in the scene are being considered as shadow casters. So if you have an object behind the camera that's casting a shadow in front of the camera then the shadow will not be visible. Depending on your application this may not be an issue, but it's something to keep in mind. 

Share this post


Link to post
Share on other sites
That part of the code is actually changed from the sample. There's a comment on MJP's blog where a user uploaded a changed ComputeFrustum function that is supposed to reduce jitter that occurs when the camera is moving. The code is here: http://pastebin.com/Yn5SVPUP. Is that code actually wrong? Should I just use MJP's original version instead?

Share this post


Link to post
Share on other sites

Ok cool, one potential issue down smile.png

 

Another thing I noticed, In your original images the shadow maps look upside down. This could be a slimDX thing? And maybe you're compensating for that? So maybe try this... In your .fx file invert the .y element of your texture coordinates when you are checking to see if a pixel is in shadow or not. So somewhere in there you will have something like:

tex2D(ShadowMapSampler, vShadowTexCoord)

 

Try:

 

vShadowTexCoord.y = 1.0 - vShadowTexCoord.y;
tex2D(ShadowMapSampler, vShadowTexCoord)
Edited by Nyssa

Share this post


Link to post
Share on other sites

Shadow mapping and skeletal animation systems are the best way to get a headache. :)

 

Can't tell you what is wrong, but I would start with some debugging setup which could help pinning down the bug.

1. Start with one cascade as Nyssa said.

2. Place some recognizable shadow caster , a sphere, a  box and a pyramid. The latter is really helpful to check if the texture is upside-down.

3. Draw the full shadow-camera frustums as object.

Share this post


Link to post
Share on other sites
The sampling Y position is already inverted. If I remove it, the shadows seem to look even weirder.

Heres what it looks like with 1 cascade with a recognizable object in the scene:

[attachment=13682:RuinValor 2013-02-17 17-12-36-25.png]

And again, with the camera moved to the side a bit (the light position hasn't changed):

[attachment=13685:RuinValor 2013-02-17 17-12-48-57.png]

Edit: I should probably note, those lighter colored shadows directly beneath the blocks are static shadows, completely unrelated to this. Ignore them. Edited by Telanor

Share this post


Link to post
Share on other sites

When you move the camera, the shadow seems to shift. Therefor I guess, that your back transform works up to the projection part (viewport.

Well, check your back transformation pipeline (view->world->light space->light frustum), something like this

C = camera transform (in world space)
L = shadow camera transform (in world space)

Assumption: you got a pixel position pos_v  in view space reconstructed from the framebuffer

// 1. View to world space
pos_w = C * pos_v // NOT the inverse camera transform

// 2. world to light space
pos_l = inverse(L) * pow_w

// 3. Project on shadow frustrum
pos_final = L_proj * pos_l

// 4. get shadow texel 
shadow_texel  = 2dShadow( shadow_map, pos_final)


Edited by Ashaman73

Share this post


Link to post
Share on other sites
I'm not very good with transforming things from one space to another, so I'll try to go through this one by one and see if I understand it.
// Reconstruct view-space position from the depth buffer
float pixelDepth = DepthMap.Sample(DepthMapSampler, input.TexCoord).r;
float4 position = float4(pixelDepth * input.FrustumCornerVS, 1.0f);
The comment from the sample says that position is going to be in view-space, which fits the assumption your equation makes. I've never seen this kind of position reconstruction before and don't really understand it, so I can only assume it's doing what it says.

The next part is:
float4x4 inverseLVP = mul(InverseView, lightViewProjection);
float4 positionLight = mul(position, inverseLVP);
InverseView here is the inverse of the player's camera's view matrix. lightViewProjection is the View * Projection matrix for the cascade camera this pixel is in. Broken down, mul(position, InverseView) matches your step 1. inverse(L) should be the view matrix for the cascade camera and L_proj the projection matrix. So in mine it's combined into 1 matrix. So I think it is going through the right transforms, right? Edited by Telanor

Share this post


Link to post
Share on other sites

This process seems correct, but I think that one or more matrices are wrongly calculated. When looking at your screenshots, the sign is pointing to the left on your shadowmap (middle). The lighting on the sign in the screenshot supports this (light coming from the right side of the screenshot), therefor the shadow should fall to the left, but it falls to the right.

 

Check and debug your InverseView first, it should be used to put your pixels into worldspace. You could try to colorencode the world position relative to a reference point and a scale factor to check it, it should be stable, even if you rotation your camera. Testshader:

 

world_position = InverseView * view_position;
SCALE = 1.0/100.0; // units ?
color_encoded_position = (world_position - camera_position) *  SCALE;
color_encoded_position = clamp(0.0,1.0,color_encoded_position* 0.5 + 0.5) 

output_color = color_encoded_position

This should color the world in a moving, axis aligned 3d cube centered at your camera. Rotation should not effect the coloring, and movement should shift the cube.

Share this post


Link to post
Share on other sites
I assume camera_position means the player's camera position? Not really sure what I'm looking for here, but this is what it came out as:

[attachment=13689:RuinValor 2013-02-18 02-41-34-17.png]
[attachment=13690:RuinValor 2013-02-18 02-41-45-96.png]
[attachment=13691:RuinValor 2013-02-18 02-41-50-60.png]

Share this post


Link to post
Share on other sites

Yes, the camera position, what happens if you stand still and rotate the camera only ? In this case the "terrain texture" should not change, if it change, your InverseView matrix is most likely broken.

 

An other test, if you only move the camera along the lookat,right axis (no rotation)  the color pattern should stay the same (like a projected texture pointing along the up-vector centered at the camera).

Edited by Ashaman73

Share this post


Link to post
Share on other sites
Those are shots from just the rotation of the camera. So I guess that means the InverseView is broken...? Not really sure how that can be the case, there are other places where the InverseView is used and they don't have any problems.

Share this post


Link to post
Share on other sites

The inverse view matrix should convert your view space into worldspace and in world space the positions of your pixels are fix, therefor the rotation of your camera (camera position is fix) should not change them. So, what is your view matrix and how do you calculate its inverse ?

Share this post


Link to post
Share on other sites

There have been many suggestions here already so I'm just going out on a limb to say this- have you accounted for XNA's coordinate system orientation? You said that you are adapting an XNA code sample for use in DirectX/SharpDX. XNA uses a right-handed coordinate system while DirectX's is left-handed, so the Z values are flipped the other way around. This could cause odd behavior in rotation matrices when you are applying XNA code as-is.

Share this post


Link to post
Share on other sites
Here's how I do the calculation:
world = Matrix.RotationYawPitchRoll(Yaw, Pitch, 0);
world.TranslationVector = pos;

Matrix.Invert(ref world, out view);
The world matrix is sent to the shader as the InverseView matrix. As for the coordinate system, we originally used XNA and then switched over to SharpDX, so to avoid having to change a ton of stuff, we stuck with the XNA system where Y=up and we use right-handed matrices.

Share this post


Link to post
Share on other sites

You invert the camera orientation, but not the view matrix. The view matrix is already the inverted camera orientation, therefor you need to inverting it a second time. So , try just this

view = Matrix.RotationYawPitchRoll(Yaw, Pitch, 0);
view.TranslationVector = pos;

(invert*invert=identity)

Edited by Ashaman73

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this