I am designing matrix functions for a right handed coordinate system where x is right, y is forward, and z is up.
Currently, I am using matrix code taken from a right handed system where x is right, y is up, and z is backward. I have not modified this code except for yaw pitch and roll which seems to be working fine via a 90 degree clockwise x axis rotation applied to it. Although the game terrain appears to look normal, the camera starts facing down instead of forward and movement of the camera position is very odd.
I am thinking that some modifications to the perspective and lookat matrices are needed. I have tried a few modifications such as the same 90 degree rotation given to yaw pitch and roll, but none of these modifications have helped much and they usually make things worse.
Here are the functions for perspective and lookat matrices that I am currently using:
Does anyone have any ideas on how these functions can be corrected?
For starters it seems that height might be better off in a z position instead of a y position in perspective, and that given the different direction of up, z and y could be swapped in some way in lookat. However simple swaps don't seem to help much, so I must be missing something.
You talk about 90° around x, I see your LookAt function and the very first question obviously has to be "what are you passing as 'up'"? If your answer is "Why, [0,1,0] of course" I will have to make a very annoying "wrong answer" buzzer sound and ask you what you think the result of [0,1,0] x [0,1,0] is going to be (hint: it's a vector without any direction at all).
If you actually did make sure to use a valid up-vector and everything is looking right before you start moving, I'd say your projection matrix is probably fine.
I was using the 90 degree around x idea because the conversion from the right handed system that I got my matrix algorithms from, to my right handed system (where z is up) is a 90 degree clockwise rotation around the x axis. However, I am not currently using that.
The up vector I am passing in is [0, 0, 1].
Update:
I'm still having issues, just slightly different ones.
I have rewritten my yaw pitch and roll function based on the relation of yaw pitch and roll to my coordinate system instead of rotating the matrix from the other system.
As a result, up is now down and yaw pitch and roll is working fine if and only if you remain in the up is down orientation. In addition, movement to the right results in movement to the left and vice versa regardless of what up down orientation you place yourself into. Also, you now start facing the correct direction of forward (positive y).
Here is the yaw pitch and roll function that I am using. There are some pretty obvious optimizations to it that I'll implement once things are working properly.
I reordered the cross product between the up and zAxis. As a result, I start facing the correct direction with up in the correct direction as well. Yaw pitch and roll are now inverted, so if I try to look left, I look right instead. My forward, backward, left and right movements relative to the camera are correct unless i angle the camera down. When I angle the camera down these movements will reverse after a certain threshold.
Any help would be much appreciated. I've been stuck on this problem for far too long now.
Edit: Negating the yaw and pitch values gives correct camera rotations, however movement of the camera still has problems when you angle the camera away from the z axis.