I've been trying to get ray picking working correctly for a few days now and have been unable to find a solution via these forums or the interwebs. I have ray picking implemented correctly for objects that do not use Matrix.Scaling in their world transform. When no scaling is applied the ray picking works perfectly, as seen below. I've used a small black X to show approximately where the cursor is, as Print Screen doesn't capture the cursor. There are two meshes in this screenshot. There is the outer cabinet (The box), and the two drawers, which use the same mesh. The bottom one simply has a scaling matrix applied. The mesh that is picked is slightly reddened.
However, with the scaling matrix applied to the world transform the ray picking obviously doesn't work correctly. Same idea as the above image as far as cursor and meshes. Obviously the outside cabinet should be picked but instead the drawer inside is. There is no overlap between the meshes.
To do the ray picking, I get the origin and direction vectors of the ray from screen coordinates via the following code.
private Vector3 getRayFromScreen(Vector2 ScreenCoordinates, Viewport Viewport, Matrix WorldTransform, out Vector3 RayDirection)
//First, find where the ray will start in screen space.
Vector3 rayOrigin = Vector3.Unproject(new Vector3(ScreenCoordinates.X, ScreenCoordinates.Y, 0), Viewport, device.Transform.Projection, device.Transform.View, WorldTransform);
//Project the ray straight out into world space to get its end.
Vector3 rayEnd = Vector3.Unproject(new Vector3(ScreenCoordinates.X, ScreenCoordinates.Y, 1), Viewport, device.Transform.Projection, device.Transform.View, WorldTransform);
//Get the direction by subtracting the end point from the start point and then normalizing.
RayDirection = Vector3.Normalize(rayEnd - rayOrigin);
The WorldTransform argument is simply the world transform of the mesh, so the Vectors are projected into model space. I then use Mesh.Intersect to check for intersection. I compare the IntersectInformation.Dist values to find the one with the shortest distance (Closest to the camera) and that is the mesh that is then considered "picked." It is then flagged and colored accordingly by the shader.
I can't figure out what isn't right here. As far as I know, the matrices are multiplied in the right order... Scaling * Rotation * Translation. Should I not use the mesh world transform and instead use Matrix.Identity then attempt to transform the ray into model space?