Sign in to follow this  
Ripiz

Mouse ray/picking trouble

Recommended Posts

Ripiz    539
Hello,

I am using Sony's Vectormath from Bullet Physics download and DirectX 11. I'm trying to add mouse picking, but I'm having some troubles creating ray. Maybe someone could help me out? Here's my code:
[CODE]
const Matrix4 &Proj = gCamera.GetProjMatrix();
const Matrix4 &ViewInv = inverse(gCamera.GetViewMatrix());
const Vector3 MouseRay(
(((2.0f * MousePos[0]) / Viewports[0].Width) - 1) / Proj[0][0],
-(((2.0f * MousePos[1]) / Viewports[0].Height) - 1) / Proj[1][1],
1.0f
);
const Vector3 &Direction = normalize(Vector3(
dot(ViewInv[0].getXYZ(), MouseRay),
dot(ViewInv[1].getXYZ(), MouseRay),
dot(ViewInv[2].getXYZ(), MouseRay)
));
const Vector3 &Start = ViewInv[3].getXYZ();
[/CODE]

[i]Start[/i] seems to be correct, however direction is not correct. [i]Direction[1][/i] is positive (y axis), but my camera is facing down, so it should have been negative.

Thank you in advance.

Share this post


Link to post
Share on other sites
Ripiz    539
Tried to untransform screen pixel using inverse Proj*View matrix, but it's not working either. Direction is more-or-less correct; if my mouse is at the center of screen it works correctly, however when I move mouse away ray's direction doesn't change fast enough, that is my mouse is near the edge of window, but ray gets casted just a little bit away from screen's center. Code:
[CODE]
const Vector4 MouseRay(
MousePos[0] - Viewports[0].Width / 2,
Viewports[0].Height / 2 - MousePos[1],
gCamera.mFar,
gCamera.mFar
);
const Matrix4 &ProjViewInv = inverse(gCamera.GetProjViewMatrix());
const Vector3 &Direction = normalize((ProjViewInv * MouseRay).getXYZ());
[/CODE]
Anyone knows how to get it working correctly?

Thank you in advance.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this