[color=#1C2837][size=2]Project the coordinates of the labels (I assume you know what I mean by that)
[/quote]

This is where I think I am going wrong:

- If I draw the labels just using an orthographic projection they do not move but will always face the camera (like a GUI) when the camera rotates.
- If I transform them by the camera's view matrix with 0,0,0 for the translation they will be in the correct location but they will not face the camera.

Can you please explain what you mean by "project the coordinates of the labels"?

look into "world coordinates to screen coordinates"[/quote]

Can you please explain why I should use this method? I've seen it used as a Unity add on here but I have solved it by doing the following:




// Always use the active camera to get the label to face the current camera
Camera camera = game.ActiveCamera;
Viewport viewport = game.GraphicsDevice.Viewport;

// Scale
//Matrix worldMatrix = Matrix.CreateScale(0.5f);
Matrix worldMatrix = Matrix.CreateScale(50f);

// Rotate
worldMatrix *= camera.ViewMatrix;

// Set its last column (the translation part) to (0, 0, 0, 1)
worldMatrix.M41 = 0f;
worldMatrix.M42 = 0f;
worldMatrix.M43 = 0f;
worldMatrix.M44 = 1f;

// The transpose of view matrix to make the vertices face the current camera
Matrix.Transpose(ref worldMatrix, out worldMatrix);

// Translate
worldMatrix.M41 = ObjectPosition.X;
worldMatrix.M42 = ObjectPosition.Y;
worldMatrix.M43 = ObjectPosition.Z;

vertices[0] = Vector3.Transform(corners[0], worldMatrix);
vertices[1] = Vector3.Transform(corners[1], worldMatrix);
vertices[2] = Vector3.Transform(corners[2], worldMatrix);
vertices[3] = Vector3.Transform(corners[3], worldMatrix);


Matrix orthogonal = Matrix.CreateOrthographic(
viewport.Width,
viewport.Height,
-viewport.Width,
viewport.Width
);

Matrix viewMatrix = camera.ViewMatrix;

// Set its last column (the translation part) to (0, 0, 0, 1)
viewMatrix.M41 = 0f;
viewMatrix.M42 = 0f;
viewMatrix.M43 = 0f;
viewMatrix.M44 = 1f;

game.WireShapeDrawer.Begin(viewMatrix, orthogonal);

game.WireShapeDrawer.DrawLine(vertices[0], vertices[1], Color.White);
game.WireShapeDrawer.DrawLine(vertices[1], vertices[3], Color.White);
game.WireShapeDrawer.DrawLine(vertices[3], vertices[2], Color.White);
game.WireShapeDrawer.DrawLine(vertices[2], vertices[0], Color.White);

game.WireShapeDrawer.End();


The video of the results are shown here

I hope that someone can enlighten me why conversion to screen space coordinates are often used instead.