I'm working on some basic AI for a fighter game, and I'm trying to get the ships to fly into a formation. To calculate the amount of torque I need to apply to each ship so that it turns towards the desired heading I'm using a PID controller very similar to the one in the first answer here: http://answers.unity3d.com/questions/199055/addtorque-to-rotate-rigidbody-to-look-at-a-point.html
So, I have the current heading of the ship, and the desired heading, and I need to pass an error value into the PID controller so that it can correct the heading.
In the linked example, the poster uses the cross product of the desired heading and current heading. This works perfectly if the angle between the headings is < 90 degrees, if it's greater than 90 degrees then the cross product is flipped and the PID controller starts correcting to the opposite direction than I want!
I'm struggling to figure out what vec3 error value will work in all cases, even if there is a large angle between the desired and actual heading. Any ideas?