Advertisement Jump to content

DX11 D3D11 Picking Problem

Recommended Posts

Hi guys, when I do picking followed by ray-plane intersection the results are all wrong. I am pretty sure my ray-plane intersection is correct so I'll just show the picking part. Please take a look:


// get projection_matrix
DirectX::XMFLOAT4X4 mat;
DirectX::XMStoreFloat4x4(&mat, projection_matrix);

float2 v;
v.x = (((2.0f * (float)mouse_x) / (float)screen_width) - 1.0f) / mat._11;
v.y = -(((2.0f * (float)mouse_y) / (float)screen_height) - 1.0f) / mat._22;

// get inverse of view_matrix
DirectX::XMMATRIX inv_view = DirectX::XMMatrixInverse(nullptr, view_matrix);
DirectX::XMStoreFloat4x4(&mat, inv_view);

// create ray origin (camera position)
float3 ray_origin;
ray_origin.x = mat._41;
ray_origin.y = mat._42;
ray_origin.z = mat._43;

// create ray direction
float3 ray_dir;
ray_dir.x  = v.x * mat._11 + v.y * mat._21 + mat._31;
ray_dir.y  = v.x * mat._12 + v.y * mat._22 + mat._32;
ray_dir.z  = v.x * mat._13 + v.y * mat._23 + mat._33;


That should give me a ray origin and direction in world space but when I do the ray-plane intersection the results are all wrong.

If I click on the bottom half of the screen ray_dir.z becomes negative (more so as I click lower). I don't understand how that can be, shouldn't it always be pointing down the z-axis ?

I had this working in the past but I can't find my old code :(

Please help. Thank you.

Edited by Endemoniada
Typo in code.

Share this post

Link to post
Share on other sites

was this a typo? (using an "m" matrix, i don't see it anywhere else in that code, i'm sure you meant "mat")

float3 ray_origin;
ray_origin.x = m._41;
ray_origin.y = m._42;
ray_origin.z = m._43;

Also it's not weird that ray_dir.z would be negative the further down on the screen you click, if your camera is facing down. ray_dir and ray_origin should be in world coordinates, so the direction that ray points depends on the direction your camera is facing (as well as where on the screen you click)

Share this post

Link to post
Share on other sites

Hi iedoc, yes, that was a typo, I fixed it.


My camera is pointing down (isometric), like in Diablo. The ray from the camera position (eye) to the camera look at point is always positive z, so it's hard for me to see how the picking ray direction can have a negative z. In any case, I still can't figure out what I'm doing wrong. When I click parts of the flat ground that are in positive x and positive z they come up as negative and are totally inaccurate.


For example, when I click the ground at world coordinates (0, 0, 0) it shows ~ (-70, 0, -70) and when I click at (4, 0, 0) it shows ~ (-58, 0, -80), the values are way off, the z shouldn't change at all, and the x should only change by 4.


I'll keep trying.

Share this post

Link to post
Share on other sites

what shows (-70, 0, -70)? the ray origin? or the ray direction? If neither, make sure your getting a ray you would expect to get before anything else. is the ray origin the position of the camera correct? is the ray direction somewhat in the direction your camera is facing?

Share this post

Link to post
Share on other sites

Hi guys, I'm just getting back to this and still can't figure out what's wrong. I'll post the complete code and results:


//	view matrix

float3 camera_target = { 0.0f, 0.0f, 0.0f };

DirectX::XMVECTOR look = DirectX::XMVectorSet(camera_target.x, camera_target.y, camera_target.z, 0.0f);
DirectX::XMVECTOR eye = DirectX::XMVectorSet(camera_target.x, camera_target.y + 64.0f, camera_target.z - 64.0f, 0.0f);
DirectX::XMVECTOR up = DirectX::XMVectorSet(0.0f, 1.0f, 0.0f, 0.0f);

DirectX::XMMATRIX view_matrix = DirectX::XMMatrixLookAtLH(eye, look, up);

//	projection matrix (orthographic with same aspect as screen 16:9)

DirectX::XMMATRIX projection_matrix = DirectX::XMMatrixOrthographicLH(160.0f, 90.0f, 8.0f, 128.0f);

//	mouse/screen coordinates

float screen_width = 1920;
float screen_height = 1080;

float mouse_x = params[0];
float mouse_y = params[1];

//	picking part

DirectX::XMFLOAT4X4 mat;
DirectX::XMStoreFloat4x4(&mat, projection_matrix);

float point_x = ((2.0f * mouse_x) / screen_width) - 1.0f;
float point_y = -(((2.0f * mouse_y) / screen_height) - 1.0f);

point_x = point_x / mat._11;
point_y = point_y / mat._22;

DirectX::XMMATRIX inv_view = DirectX::XMMatrixInverse(nullptr, view_matrix);
DirectX::XMStoreFloat4x4(&mat, inv_view);

float3 ray_origin = { mat._41, mat._42, mat._43 };

float3 ray_dir;
ray_dir.x = (point_x * mat._11) + (point_y * mat._21) + mat._31;
ray_dir.y = (point_x * mat._12) + (point_y * mat._22) + mat._32;
ray_dir.z = (point_x * mat._13) + (point_y * mat._23) + mat._33;

//	normalize ray_dir (not sure if necessary)


The camera is simply above and behind the world origin, looking down and forward at it, like a basic top-down view. The screen resolution (same as render target) is 1920 x 1080.

Here are the results:

(mouse coords in screen space) : (ray direction)

(960, 540) : (0.0, -0.707, 0.707)
(970, 540) : (0.640, -0.543, 0.543)
(1000, 540) : (0.958, -0.203, 0.203)

The first set is the centre of the screen and seems fine. The second set is just 10 pixels over to the right, but look at the ray_dir.x. Look at the third set, the mouse is just 40 pixels over to the right and the ray is pretty much pointing down the x-axis.

I feel like I'm missing a step somewhere. Please help. Thank you.




Share this post

Link to post
Share on other sites

What happens if you try this?

DirectX::XMStoreFloat4x4(&mat, XMMatrixTranspose(projection_matrix));
DirectX::XMStoreFloat4x4(&mat, XMMatrixTranspose(inv_view));

Share this post

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Advertisement

Important Information

By using, you agree to our community Guidelines, Terms of Use, and Privacy Policy. is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!