I've been messing with this all day but haven't gotten any satisfactory results.

I'm using an orthographic projection for my shadow mapping and I calculate and fit the ortho to fit exactly around the view frustrum. I also rotate the ortho so the frustrum fills as much of the space as possible.

The shadows are all correct and in their normal places but when I rotate the camera or move around it gives this exceptionally obnoxious shimmering effect. At certain angles I can very well see a moire pattern as well.

What is the best way to go about this? I've tried cranking up the PCF blending and it's no good, and I've also added a bunch to the z to offset the moire pattern but it's still shimmering and looking messed up at weird angles.

Should I just fit the ortho to the view frustrum but not rotate the ortho to completely fill the space? I think the rotation is whats causing the shimmering.

I'd appreciate any advice on the subject. Thanks.