I think I have it figured out. I will post more soon if I get it working. That way, if others have problems with omni-directional shadow maps, or spot lights in general, it will hopefully help them. =)
Basically, the problem was pretty isolated to a trigonometry problem, which was good. But, it was a silly mistake and I guess being over tired and rushing, I wasn't even thinking. The following code (and previous samples) are working for a basic spot-light effect with shadow mapping in XNA. There still needs to be changes to include range-distance decay, and obviously the code can be optimized and condensed, but it's a good starter point. =)
Code changes :
(1) Added m_fInnerConeCos and m_fOuterConeCos variables, which are calculated whenever m_fOuterConeAngle and m_fInnerConeAngle change. I simply find m_fOuterConeAngle and m_fInnerConeAngle normalized values easier to type in for an editor and visualize, but the shader needs the new variables.
// Angle between light direction and cone edge. // Normalized between 0 and 1 for half-cone angle, where 0 is no angle and 1 is 90 degrees (full 180 spot-light). private float m_fInnerConeAngle = 0.0f; private float m_fOuterConeAngle = 0.0f;
private float m_fInnerConeCos = 0.0f; // Cached value. To compare to a dot product value in a spot light shader. private float m_fOuterConeCos = 0.0f;
new variables are calculated like so, if/whenever the cone's change, on the CPU in XNA: