So, I've mostly gotting everything working except for one minor detail that I can't seem to get rid off, the AABB doesn't seem big enough to cover the entire frustum and it gets a bit warped whenever I rotate the camera. I've just padded the xy with an offset until I got it to go away but that wastes too much resolution. Here's when I look down the negative z axis:

And when I rotate it a bit:

Then it goes away complete as I'm perpendicular to the z axis, and if I keep going the edges are on the other side of the screen instead

I have no clue why that happens and I havent seen anyone mentioning that you need to widen the X/Y radius to get it to go away, so I'm assuming I'm doing something wrong somewhere. Here's the complete code as of now:

void CascadedShadowTechnique::renderToShadowMap(Mesh* mesh, Mesh* secondMesh)
{
	//Get the active cameras frustum parameters
	Frustum cameraFrustum = CameraMan.getActiveCamera()->mFrustum;

	//Start off by calculating the split distances
	GLfloat cascadeSplits[MAX_SPLITS] = {};

	//Between 0 and 1
	GLfloat lambda = 1.0f;

	//min 0 max 1
	GLfloat minDistance = 0.3f;
	GLfloat maxDistance = 1.0f;

	GLfloat nearClip = cameraFrustum.mNear;
	GLfloat farClip = cameraFrustum.mFar;
	GLfloat clipRange = farClip - nearClip;

	//Change min/maxDistance to have get different sized splits
	GLfloat minZ = nearClip + minDistance * clipRange;
	GLfloat maxZ = nearClip + maxDistance * clipRange;

	GLfloat range = maxZ - minZ;
	GLfloat ratio = maxZ / minZ;

	for (unsigned int i = 0; i < MAX_SPLITS; ++i)
	{
		GLfloat p = (i + 1) / static_cast<GLfloat>(MAX_SPLITS);
		// n(f/n)^i/m
		GLfloat log = minZ * std::pow(ratio, p);
		GLfloat uniform = minZ + range * p;
		GLfloat d = lambda * (log - uniform) + uniform;
		cascadeSplits[i] = (d - nearClip) / clipRange;
	}

	for (unsigned int cascadeIterator = 0; cascadeIterator < MAX_SPLITS; ++cascadeIterator)
	{

		GLfloat prevSplitDistance = cascadeIterator == 0 ? minDistance : cascadeSplits[cascadeIterator - 1]; 
		GLfloat splitDistance = cascadeSplits[cascadeIterator];

		// Get the 8 points of the view frustum in world space
		glm::vec3 frustumCornersWS[8] =
		{
			glm::vec3(-1.0f, 1.0f, -1.0f),
			glm::vec3( 1.0f, 1.0f, -1.0f),
			glm::vec3( 1.0f,-1.0f, -1.0f),
			glm::vec3(-1.0f,-1.0f, -1.0f),
			glm::vec3(-1.0f, 1.0f, 1.0f),
			glm::vec3( 1.0f, 1.0f, 1.0f),
			glm::vec3( 1.0f,-1.0f, 1.0f),
			glm::vec3(-1.0f,-1.0f, 1.0f),
		};

		glm::mat4 invViewProj = glm::inverse(CameraMan.getActiveCamera()->mProjectionMatrix * CameraMan.getActiveCamera()->getViewMatrix());
		for (unsigned int i = 0; i < 8; ++i)
		{
			glm::vec4 inversePoint = invViewProj * glm::vec4(frustumCornersWS[i], 1.0f);
			frustumCornersWS[i] = glm::vec3(inversePoint/inversePoint.w);
		}
		// Get the corners of the current cascade slice of the view frustum
		for (unsigned int i = 0; i < 4; ++i)
		{
			glm::vec3 cornerRay = frustumCornersWS[i + 4] - frustumCornersWS[i];
			glm::vec3 nearCornerRay = cornerRay * prevSplitDistance;
			glm::vec3 farCornerRay = cornerRay * splitDistance;
			frustumCornersWS[i + 4] = frustumCornersWS[i] + farCornerRay;
			frustumCornersWS[i] = frustumCornersWS[i] + nearCornerRay;
		}

		//Calculate the frustum center
		glm::vec3 frustumCenter = glm::vec3(0.0f);
		for (unsigned int i = 0; i < 8; ++i)
			frustumCenter += frustumCornersWS[i];
		frustumCenter /= 8.0f;


		GLfloat far = -INFINITY;
		GLfloat near = INFINITY;

		//Get the longest radius in world space
		GLfloat radius = 0.0f;
		for (unsigned int i = 0; i < 8; ++i)
		{
			GLfloat distance = glm::length(frustumCornersWS[i] - frustumCenter);
			radius = glm::max(radius, distance);
		}
		radius = std::ceil(radius);

		//Create the AABB from the radius
		glm::vec3 maxOrtho = frustumCenter + glm::vec3(radius, radius, radius);
		glm::vec3 minOrtho = frustumCenter - glm::vec3(radius, radius, radius);

		//Just checking when debugging to make sure the AABB is the same size
		GLfloat lengthofTemp = glm::length(maxOrtho - minOrtho);
		GLfloat testing = radius*2.0f;

		//Calculate the viewMatrix from the frustum center and light direction
		glm::vec3 lightDirection = frustumCenter - glm::normalize(glm::vec3(-0.447213620f, -0.89442790f, 0.0f));
		lightViewMatrix = glm::lookAt(lightDirection, frustumCenter, glm::vec3(0.0f, 1.0f, 0.0f));

		//Get the AABB in light view space
		maxOrtho = glm::vec3(lightViewMatrix*glm::vec4(maxOrtho, 1.0f));
		minOrtho = glm::vec3(lightViewMatrix*glm::vec4(minOrtho, 1.0f));



		//Store the far and near planes
		far = maxOrtho.z;
		near = minOrtho.z;

		lightOrthoMatrix = glm::ortho(maxOrtho.x, minOrtho.x, maxOrtho.y, minOrtho.y, near, far);

		// Create the rounding matrix, by projecting the world-space origin and determining
		// the fractional offset in texel space
		glm::mat4 shadowMatrix = lightOrthoMatrix * lightViewMatrix;
		glm::vec4 shadowOrigin = glm::vec4(0.0f, 0.0f, 0.0f, 1.0f);
		shadowOrigin = shadowMatrix * shadowOrigin;
		shadowOrigin = shadowOrigin * mShadowMapSize / 2.0f;

		glm::vec4 roundedOrigin = glm::round(shadowOrigin);
		glm::vec4 roundOffset = roundedOrigin - shadowOrigin;
		roundOffset = roundOffset *  2.0f / mShadowMapSize;
		roundOffset.z = 0.0f;
		roundOffset.w = 0.0f;

		glm::mat4 shadowProj = lightOrthoMatrix;
		shadowProj[3] += roundOffset;
		lightOrthoMatrix = shadowProj;

		// Store the split distance in terms of view space depth
		const float clipDist = cameraFrustum.mFar - cameraFrustum.mNear;
		cascadeSplitArray[cascadeIterator] = cameraFrustum.mNear + splitDistance * clipDist;

		cascadedMatrices[cascadeIterator] = lightOrthoMatrix * lightViewMatrix;

		//Now finally render the scene into FBO
		ShaderMan.bindShader(SHADOW_DEPTH_PASS_SHADER);
		glBindFramebuffer(GL_FRAMEBUFFER, mCascadedShadowFBO);
		glViewport(0, 0, mShadowMapSize, mShadowMapSize);

		glFramebufferTextureLayer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, mCascadedTextureArray, 0, cascadeIterator);

		glClear(GL_DEPTH_BUFFER_BIT);
		glEnable(GL_DEPTH_TEST);
		glEnable(GL_DEPTH_CLAMP);
		glCullFace(GL_FRONT);

		glm::mat4 lightViewProjection = lightOrthoMatrix * lightViewMatrix;
		glUniformMatrix4fv(ShaderMan.getActiveShader().getUniformLocation("lightViewProjectionMatrix"), 1, GL_FALSE, &lightViewProjection[0][0]);

		mesh->renderDepth();

		secondMesh->renderDepth();

		glBindFramebuffer(GL_FRAMEBUFFER, 0);
		ShaderMan.unbindShader();
		glDisable(GL_DEPTH_CLAMP);
	}
}

I'm pretty confident that the frustums are calculated correctly, but I can't find any reason for what's happening.

Okay, so since I create the matrix using:

lightOrthoMatrix = glm::ortho(maxOrtho.x, minOrtho.x, maxOrtho.y, minOrtho.y, near, far);

That means that if I rotate the camera 45 degrees and move to the left/right, the orthographic matrix will start moving either up and down or sideways. These videos and my amazing paint image should explain it:

Correct movement:

Incorrect movement:

You can see how the map gets cut off in the second video since it's no longer moving with the camera.

Also made this image to sort of illustrate the point:

if I rotate the camera then the min/max x and y aren't the same anymore. However if I start rotating the points along with the camera I get light shimmering again since the shadow map changes size. I can sort of alleviate the issue by doing this:

lightOrthoMatrix = glm::ortho(minOrtho.x, maxOrtho.x, minOrtho.x, maxOrtho.x, near, far);

Just changing the y values to the x values. Now I know that this is incorrect but I don't know how else to pad the frustum, everyone else seems to be doing it with just taking the radius and setting that to the max/min values and everythings perfect. I mean I guess I could leave it like this but I'll be wasting a lot of resolution and it's an incorrect solution

Using SDSM to have tighter projection you can use the AABB to not waste space because shimmering is not visible.

If you simply use a fixed distance, then you will see the shimmering and you need to have rotation invariance using bounding sphere.

if I rotate the camera then the min/max x and y aren't the same anymore.

If I understand you correctly, that isn't actually true. When you add the radius vector to the frustum center here

//Create the AABB from the radius
glm::vec3 maxOrtho = frustumCenter + glm::vec3(radius, radius, radius);
glm::vec3 minOrtho = frustumCenter - glm::vec3(radius, radius, radius);

you insure, that it is enclosed in a sphere that doesn't change, no matter how the frustum is oriented. The outer square in the image below is the shadowmap, C is frustum center and R is radius:

In this respect your videos look correct though: the shadow map doesn't change size or orientation when the camera rotates, so that's good.

A couple of things look suspicious to me. First, this line:

lightOrthoMatrix = glm::ortho(maxOrtho.x, minOrtho.x, maxOrtho.y, minOrtho.y, near, far);

I believe, it should be min first, max second. Don't think this is the issue, but the resulting matrix probably flips the scene, so I'd rather avoid that.

Second, why do the shadowmaps you draw onscreen look rectangular? Is this just a projection issue? The movement in the second video looks fine to me, I'd rather say that the shadowmap doesn't cover the view frustum entirely. I don't see any issues with the code though, so...

...finally, it could be useful to hack in some debug features to better understand, what is happening. Drawing the scene from the point of view of the light is one option, rendering the view frustum and shadow bound edges to the frame buffer as lines is another. The latter helped me deal with the similar issues in my code a lot, since it becomes obvious, whether your frustum and shadow bound are properly aligned and have expected dimensions.

lightOrthoMatrix = glm::ortho(minOrtho.x, maxOrtho.x, minOrtho.x, maxOrtho.x, near, far);
Just changing the y values to the x values. Now I know that this is incorrect but I don't know how else to pad the frustum

Shouldn't the minOrtho.x and minOrtho.y (maxOrtho.x and maxOrtho.y) actally be equal? Since you position your light at the frustumCenter, minOrtho and maxOrtho should just be offsets from the origin in the light view space, after you transform them by the lightViewMatrix (see image below). And since the shadow map bound is square, their x and y components should be equal. There actually may be an error somewhere, if they are not.



That needn't always be true, though. Another valid approach is to always position a directional light source at the origin and encode the offset into minOrtho and maxOrtho values. If that were the case, their x and y components could have arbitrary values:

Thanks for the replies!

I believe, it should be min first, max second. Don't think this is the issue, but the resulting matrix probably flips the scene, so I'd rather avoid that.

The only reason I changed that was because I noticed that the depth maps were flipped when I used min/max.x and min/max.y.

Second, why do the shadowmaps you draw onscreen look rectangular? Is this just a projection issue? The movement in the second video looks fine to me, I'd rather say that the shadowmap doesn't cover the view frustum entirely. I don't see any issues with the code though, so...

There might be, the bottom left depth map look like a cube when I use the x values for all the corners and rectangular when I use the x and y values like this:

Shouldn't the minOrtho.x and minOrtho.y (maxOrtho.x and maxOrtho.y) actally be equal? Since you position your light at the frustumCenter, minOrtho and maxOrtho should just be offsets from the origin in the light view space, after you transform them by the lightViewMatrix (see image below). And since the shadow map bound is square, their x and y components should be equal. There actually may be an error somewhere, if they are not.

This might be the case, but the radius is always the same in world space, and then the points get projected to light space so they're different and since I use the light space coordinates for the bounding box they should be inequal?

And I'm pretty bad at explaining things but what I was getting at with the videos is that. In the first video the camera looks along the negative Z axis, so when I move right to left I move along the X axis. Which means that the depth map also moves left to right along the X axis.

However if I rotate the camera 45 degrees like in the second video, if I then move left and right I now move along the Z axis which is the same as moving along the Y axis on the depth map (if you look imagine looking at everything from above).

That's where the "problem" lies, the light AABB gets cut off exactly as the view frustum does in the first video because they're moving in the same directions. But in the second video when I move left/right the depth maps move up and down.

Yeah.... I hope some of that makes sense. But yeah since the orthographic views are rectangular I'm probably doing something wrong like you guys said. It's just weird because the algorithm is just

1. Get the frustum corners in world space

2. Find the longest radius

3. Take the center +- radius to get the min/max

Then multipy the min /max with the light view matrix and you should be good to go.