ok.
I would first suggest that you forget about 3D until you understand the math. I would make this game as a top-down 2D shooter to start. You'll need way less linear algebra to make it work, and it will force you to learn some anyway.
but here's the deal in 3D, and it's not as trivial as it seems on the face of things. I'm also sure i've omitted some steps or gotten a couple things a little wrong... been a long time since i've implemented this stuff from scratch.
For simplicity, make it so that no units in your game can ever have roll, only pitch and yaw. Also clamp pitch so that they can never look more than just less than 90 degrees up or down. This should save you from having to deal with gimbal lock.
Math tools you will need:
Know what a vector is
Know how to take dot products and cross products of 2 vectors
know how to normalize a vector
Know what a matrix is
Know how to construct a rotation matrix from euler angles
Know how to multiply a matrix by a vector
All the vector stuff is pretty trivial. Matrices are more complicated but you can get away with just looking up implementations.
1) Find the current Enemy look vector:
Matrix rotMatrix = rotMatrixEuler( e_ang_x, e_ang_y, e_ang_z );vec3 lookVector = rotMatrix * defaultLookVector;lookVector.normalize();
2) Find the Desired facing vector
vec3 desiredVector = target.position - enemy.position;
3) Calculate delta yaw
//zero out z b/c we're just interested in yawvec3 lookNoZ( lookVector.x, lookVector.y, 0 );vec3 desiredNoZ( desiredVector.x, desiredVector.y, 0 );lookNoZ.normalize();desiredNoZ.normalize()float deltaYaw = dotProduct( lookNoZ, desiredNoZ );
4) calculate delta pitch
//create a rotation matrix using the new total yaw and the old pitch//post-multiply that matrix by the default facing vector//this will create a new vector that is facing the enemy in the XY-plane//but which is angled at the current pitch//take the dot product of that vector and the desiredVector to find an angle//this angle will be your delta pitch
fun...
-me