Hi,

I am trying to implement correct user experience for rotation tool in my game engine. I want it visually behave like the same tool in Maya or Unity.

When I rotate an object, rotation tool should also rotate, BUT its axis should always be on the near camera plane and never go to another side of the tool like described in 2 first attached screenshots from Unity. You can see here that X axis (red) go to the up part of the tool instead of the back. The same for Y and Z axis.

Currently I implement something similar, but my code has huge limitation - it gave me correct quaternion for rotation, BUT to have correct axis alignment I must rewrite my existing tool rotation, so I cannot accumulate rotation and I cannot implement correct visual experience for tool. (See next 2 screenshots). As you can see there is no difference between visual tool representation despite I rotate an object itself.

Here is code I am using currently:

/// <summary>
      /// Calculate Quaternion which will define rotation from one point to another face to face
      /// </summary>
      /// <param name="objectPosition">objectPosition is your object's position</param>
      /// <param name="targetPosition">objectToFacePosition is the position of the object to face</param>
      /// <param name="upVector">upVector is the nominal "up" vector (typically Vector3.Y)</param>
      /// <remarks>Note: this does not work when objectPosition is straight below or straight above objectToFacePosition</remarks>
      /// <returns></returns>
      public static QuaternionF RotateToFace(ref Vector3F objectPosition, ref Vector3F targetPosition, ref Vector3F upVector)
      {
         Vector3F D = (objectPosition - targetPosition);
         Vector3F right = Vector3F.Normalize(Vector3F.Cross(upVector, D));
         Vector3F backward = Vector3F.Normalize(Vector3F.Cross(right, upVector));
         Vector3F up = Vector3F.Cross(backward, right);
         Matrix4x4F rotationMatrix = new Matrix4x4F(right.X, right.Y, right.Z, 0, up.X, up.Y, up.Z, 0, backward.X, backward.Y, backward.Z, 0, 0, 0, 0, 1);
         QuaternionF orientation;
         QuaternionF.RotationMatrix(ref rotationMatrix, out orientation);
         return orientation;
      }

And I am using some hack to correctly rotate all axis and keep them 90 degrees to each other:

private void TransformRotationTool(Entity current, Camera camera)
        {
            var m = current.Transform.GetRotationMatrix();
            if (current.Name == "RightAxisManipulator")
            {
                var rot = QuaternionF.RotateToFace(current.GetRelativePosition(camera), Vector3F.Zero, m.Right);
                rot.Z = rot.Y = 0;
                rot.Normalize();
                current.Transform.SetRotation(rot);
            }

            if (current.Name == "UpAxisManipulator")
            {
                var rot = QuaternionF.RotateToFace(current.GetRelativePosition(camera), Vector3F.Zero, m.Up);
                rot.X = rot.Z = 0;
                rot.Normalize();
                current.Transform.SetRotation(rot);
            }

            if (current.Name == "ForwardAxisManipulator")
            {
                var rot = QuaternionF.RotateToFace(current.GetRelativePosition(camera), Vector3F.Zero, m.Forward);
                rot.X = rot.Y = 0;
                rot.Normalize();
                current.Transform.SetRotation(rot);
            }

            if (current.Name == "CurrentViewManipulator" || current.Name == "CurrentViewCircle")
            {
                var billboardMatrix = Matrix4x4F.BillboardLH(
                    current.GetRelativePosition(camera),
                    Vector3F.Zero,
                    camera.Up,
                    camera.Forward);
                var rot = MathHelper.GetRotationFromMatrix(billboardMatrix);
                current.Transform.SetRotation(rot);
            }
        }

As you can see I am zeroing 2 of 3 axis and renormalize quaternion to keep axis perpendicular to each other.

And when I try to accumulate rotation, I am receiving completely incorrect result.

On the last image you see what happening with my tool when I try to apply rotation to face with some basic rotation.

This issue is driving me crazy. 

Could anyone help me to implement correct behaviour? 

No one from over 200 viewers cannot help me with this issue?