I am implementing shadow mapping for my university homework, unfortunately on top of a very primitive rendering engine.
Fortunately though, the output of the engine is rather simple: renderer receives a list of meshes + their modelview matrix, with no hierarchy or anything. Just setup projection, push & load modelview, draw, pop modelview. The rendering itself is a bit complex (skeletal animation, skinning, shaders etc), but that's not the issue.
The engine provides following matrices:
1) for camera (that itself moves around the scene) I can get both it's location matices (wolrdToLocal and localToWorld) and it's "view" matrix (the one that it receives from "lookAt" method).
2) for objects, I receive a localToWorld matrix (which was sufficient for rendering), and I can get worldToLocal as well.
3) for light source, I have it's worldToLocal, localToWorld, projection and "view" matrix (the one from gluLookAt)
drawing (now) looks like this:
1) I set up PROJECTION via gluPerspective
2) I load camera's worldToLocal*view as MODELVIEW
for each mesh (in MODELVIEW mode):
1) I pushMatrix();
2) multiply by it's localToWorld;
What is the recipe for texture matrix I need to set as shadowmap's texture matrix, so I can use it in step 3 (drawing) in fragment shaders?
I got some minor success with trying textureMatrix = bias * lightModelView * lightProjection, where
1) lightModelView = result of gluLookAt using world coordinates for light origin and directon
2) lightProjection = result of gluPerspective.
but it works only to some models, namely those with worldToLocal = ID. If model is even only scaled, the shadow is places wrong (scale > 1, shadow is too big and too far from 0,0,0, and scale < 1, shadow is too small and to close to 0,0,0).
Can someone help me out? I am dying trying all combinations here, and tutorial I used does some weird stuff with a basic translation, while the engine implements way more operations, and I cannot rewrite that.