First of all, I am attempting to do this in C# (For Playstation Mobile), using OpenGL style Column-Major matrices in a Right-Handed coordinate system but obviously the math will work in any language.
My projection matrix has a Near plane at 1.0, Far plane at 1000, FOV of 45.0 and Aspect of 1.7647.
I want to get my planes in World-Space, so I build my frustum from the View-Projection Matrix (that's projectionMatrix * viewMatrix). The view Matrix is the inverse of the camera's World-Transform.
The problem is; regardless of what I tweak, I can't seem to get a correct frustum. I think that I may be missing something obvious.
Focusing on the Near and Far planes for the moment (since they have the most obvious normals when correct), when my camera is positioned looking down the negative z-axis, I get two planes facing in the same direction, rather than opposite directions. If i strafe my camera left and right (while still looking along the z axis) the x value of the normal vector changes.
Obviously, something is fundamentally wrong here; I just can't figure out what - maybe someone here can?
Applying the algorithm to just the Projection matrix (i.e. model and view matrices are identity) I get the Near and Far planes returned as the following; Far: normal[0,0-1], distance 1 Near: normal[0,0,-1], distance -0.3328891
Can someone tell me if these seem correct? Using the algorithm with just the Projection Matrix should return the planes in View Space (camera space)
When multiplying in the view matrix, all the plane normals seem to point towards the origin, rather than towards the desired center of the frustum...