Well I could, but then I would have to calculate 8 intersecting points of those planes. I want to know if perhaps I can get the clip space cube frustum transformed into world space.

So I transform the 8 vertices (-1, -1, 0 ,1) (1, -1, 0, 1) (1,1,0, 1) (-1, 1 ,0, 1), (-1, -1, 1, 1) (1, -1, 1, 1) (1,1,1, 1) (-1, 1 ,1, 1) using the inverse view-transform? Then forward into light shadow map camera space, then divide by w?

To get the points that you originally asked for, yes, you would do the inverse of the view-projection matrix. Transforming these points will give you the world space locations of the 8 corners of your view frustrum. Since these are world space points, then you would need to transform them into shadow map view space and feed them to your function that you are using to create the orthogonal projection matrix.

I have never used the function that you are describing, so I'm not sure if this is exactly what you want to do - but it seems reasonable to me. You are essentially just extracting the world space bounds of your view frustrum, and then using that to construct an appropriately sized orthogonal projection matrix based around your light's position/orientation.

I am not frustum culling

So, I would normally just let you carry on with that plan since it might technically work (except for your plan on capturing things that can cast shadows into the perspective frustum without actually being in it), but then later you will be posting about shadow flickering and blocky shadows (which might be reduced by the 2 methods you mentioned, but since you also implied a certain expectation on the resulting quality, I can just say now it won’t be up to your expectations whichever way you go), and you would end up dropping this method entirely and having to do it all over again anyway, and I’d have to explain all of this then instead of now, and you would have a mess of code after all the things you had to scrap.
 
 
Once again, I repeat: Scrap the idea of using points.  And if you already have frustum culling implemented or plan to implement it, I am not sure why you chose this plan.
 
 
Set-up:

• Create the perspective matrix.  I assume you already do this.
• Extract the player’s frustum from it.
• Perform culling using this frustum.  Gather the objects the player can see into a list.
  • Since you have to perform frustum culling from the player’s perspective anyway, there is no point in trying to dodge this part.

Now you are done with the set-up.  You have an array of objects the player can see.  This is your primary input.
These objects can cast and receive shadows, but objects outside of the frustum are not added yet.
 
Gather out-of-view shadow casters:

• Using the existing array of objects, create an open-ended frustum for the directional light.  This means creating a 5-sided orthogonal frustum with left, right, top, bottom, and far planes.
  • Your directional light has a direction vector called a “forward vector”.
  • Calculate a right vector:

    
    Vector vRight = Vector( 0, 1, 0 ) × vFoward; // × = cross product.

  • Calculate an up vector after that:

    
    Vector vUp = vForward × vRight;

  • Normalize:

    
    vRight.Normalize();
    vUp.Normalize();
    // vForward assumed to be normalized already.

  • You now have right, up, and forward vectors all orthogonal to the light’s direction.
  • For each object in the previous array, perform a dot product with the right vector against its position and add the radius of its bounding sphere. Maintain the minimum and maximum values in the left and right directions. Do the same with the top/bottom values and the up vector, and with the far value and the forward vector (no near plane is calculated).
  • You now have the distances for left, right, top, bottom, and far planes. Make the 5 planes by combining those distances with the right, -right, up, -up, and forward vectors respectively. I shouldn’t need to explain that making a plane out of this is simply a matter of taking the min/max distance values you’ve just calculated in each direction and the positive or negative normals in those directions.
• You now have a 5-sided frustum with no near plane (it extends into infinity). Perform frustum culling and gather those objects into a list of objects that cast shadows into the player’s frustum.

From here on we only work with the new list of objects.  This list includes the perspective list’s objects in addition to all objects outside the perspective which can cast shadows.
 
Determine the dimensions of the orthogonal projection for the shadow map:

• Repeat the previous step 1, except that this time you use the new array of objects.  And this time include a “near” distance.
  • Optionally, you can tighten these values by (don don DON) calculating the 8 points in the perspective frustum and using dot products against those to determine max-left, max-right, max-top, max-bottom, max-far, and max-near.  And solving for 3 planes’ intersection is trivial.

    
    static LSE_INLINE LSBOOL LSE_CALL ThreePlanes( const CPlane3 &_pP0, const CPlane3 &_pP1, const CPlane3 &_pP2, CVector3 &_vRet ) {
      CVector3 vU = _pP1.n % _pP2.n;  // Cross product.
    
      f32 fDenom = _pP0.n * vU;
      if ( absf( fDenom ) <= static_cast<f32>(LSM_REAL_EPSILON) ) { return false; }  // LSM_REAL_EPSILON = 1.192092896e-07
    
      _vRet = (vU * _pP0.dist + _pP0.n.Cross( _pP1.n * _pP2.dist - _pP2.n * _pP1.dist )) / fDenom;
    
      return true;
    }

• Instead of using these distances to create planes, simply plug them into D3DXMatrixOrthoOffCenterLH().
• Render the shadow map using the new array of objects.

Your plan always creates the absolute most inefficient shadow map possible.  Using corner points should not be used to determine the directional light’s shadow size, it should only be used as an optional limiter. It’s the maximum size there can be, and thus a smaller bound should always be used if possible.
Additionally, your plan creates near/far discrepancies so vast that I wouldn’t be surprised if all you ever saw was a single slice of a part of a shadow. You would be absolutely maximizing aliasing and flickering, not to mention numerical instability in the shaders.
You need to make your shadow map based off what is actually in the scene. You need to select a near plane and a far plane as close together as possible and only barely what is necessary to render what will actually be seen.


The above sounds complicated but it’s really simple to implement and it will be close to what you will need to do in the future anyway. You would otherwise be plagued with quality issues and have to do all of this anyway.


L. Spiro