Jump to content
  • Advertisement
Sign in to follow this  
Ripiz

Mouse ray/picking trouble

This topic is 2344 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hello,

I am using Sony's Vectormath from Bullet Physics download and DirectX 11. I'm trying to add mouse picking, but I'm having some troubles creating ray. Maybe someone could help me out? Here's my code:

const Matrix4 &Proj = gCamera.GetProjMatrix();
const Matrix4 &ViewInv = inverse(gCamera.GetViewMatrix());
const Vector3 MouseRay(
(((2.0f * MousePos[0]) / Viewports[0].Width) - 1) / Proj[0][0],
-(((2.0f * MousePos[1]) / Viewports[0].Height) - 1) / Proj[1][1],
1.0f
);
const Vector3 &Direction = normalize(Vector3(
dot(ViewInv[0].getXYZ(), MouseRay),
dot(ViewInv[1].getXYZ(), MouseRay),
dot(ViewInv[2].getXYZ(), MouseRay)
));
const Vector3 &Start = ViewInv[3].getXYZ();


Start seems to be correct, however direction is not correct. Direction[1] is positive (y axis), but my camera is facing down, so it should have been negative.

Thank you in advance.

Share this post


Link to post
Share on other sites
Advertisement
Tried to untransform screen pixel using inverse Proj*View matrix, but it's not working either. Direction is more-or-less correct; if my mouse is at the center of screen it works correctly, however when I move mouse away ray's direction doesn't change fast enough, that is my mouse is near the edge of window, but ray gets casted just a little bit away from screen's center. Code:

const Vector4 MouseRay(
MousePos[0] - Viewports[0].Width / 2,
Viewports[0].Height / 2 - MousePos[1],
gCamera.mFar,
gCamera.mFar
);
const Matrix4 &ProjViewInv = inverse(gCamera.GetProjViewMatrix());
const Vector3 &Direction = normalize((ProjViewInv * MouseRay).getXYZ());

Anyone knows how to get it working correctly?

Thank you in advance.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!