Correct way of doing Mouse Picking

Graphics and GPU Programming Programming

Started by Zaphyk February 21, 2016 05:43 AM

5 comments, last by dave 8 years, 2 months ago

292

Author

February 21, 2016 05:43 AM

Hello, in the past few days i've been trying to implement Mouse Picking on my 3D game, however i am getting some strange results. Below is the code i am using. If anyone know any algorithm which works or can help me figure out why this doesnt work i will appreciate it very much.


Vector4 world = new Vector4( (2 * mouse.X) / viewport.Width -1, -(2 * mouse.Y) / viewport.Height -1, -1, 1
		 
Matrix4 invProj = Matrix4.Invert(proj);
Vector4 eye = Vector4.Transform(world, invProj);
			
world = new Vector4(eye.X, eye.Y, -1, 1);
			
Matrix4 invView = Matrix4.Invert(view);
			
Vector4 invVec = Vector4.Transform(world, invView);
			
Vector3 raydirection = new Vector3(invVec.X, invVec.Y, invVec.Z).Normalized();

Aardvajk

13,217

February 21, 2016 09:04 AM

If it helps, this is the code I use to compute a ray for picking (Direct3D9)

Ray Ray::compute(const Vec2 &pos, const Vec2 &size, const Matrix &view, const Matrix &proj)
{
    float w = size.x;
    float h = size.y;

    float sx = pos.x;
    float sy = pos.y;

    Vec3 v;
    v.x = (((2.0f * sx) / w ) - 1) / proj._11;
    v.y = -(((2.0f * sy) / h) - 1) / proj._22;
    v.z =  1.0f;

    Matrix m = inverseMatrix(view);

    Ray ray;

    ray.dir.x = v.x*m._11 + v.y*m._21 + v.z*m._31;
    ray.dir.y = v.x*m._12 + v.y*m._22 + v.z*m._32;
    ray.dir.z = v.x*m._13 + v.y*m._23 + v.z*m._33;
    ray.pos.x = m._41;
    ray.pos.y = m._42;
    ray.pos.z = m._43;

    ray.dir = normalizeVector(ray.dir);

    return ray;
}

pos is the mouse position on the screen, size is the width/height of the viewport. The ray returned has pos and dir and is in world frame. Matrix is just a typedef for D3DXMATRIX, Vec2 a typedef for D3DXVECTOR2 and Vec3 is a typedef for D3DXVECTOR3.

journal | website | twitter | youtube

Erik Rufelt

5,903

February 21, 2016 09:25 AM

One thing that looks like a mistake is this:

"-(2 * mouse.Y) / viewport.Height -1"

Should be +1 at the end, or your values will be between -3 and -1 instead of -1 and 1.

Norman Barrows

7,180

February 21, 2016 10:34 AM

more raypick code, based off of the dx9 raypick sample code:

http://www.gamedev.net/blog/1729/entry-2260059-raypick-test/

http://www.gamedev.net/blog/1729/entry-2260115-more-raypick-code/

http://www.gamedev.net/blog/1729/entry-2260250-more-raypicks-raycast-line-of-sight-test/

Norm Barrows

Rockland Software Productions

"Building PC games since 1989"

rocklandsoftware.net

PLAY CAVEMAN NOW!

http://rocklandsoftware.net/beta.php

Brain

19,059

February 21, 2016 11:03 AM

Here's a good way of doing picking in directx 11:

http://www.braynzarsoft.net/viewtutorial/q16390-24-picking

I used this originally but I'vemodified mine slightly to not use d3dx. It works well and is straightforward :)

Games/Projects Currently In Development:
Discord RPG Bot | D++ - The Lightweight C++ Discord API Library | TriviaBot Discord Trivia Bot

_WeirdCat_

1,512

February 21, 2016 11:44 AM



vec3 DirectionFromScreen(int x, int y, int sw, int sh, float fov, float z_near, float z_far, float aspect)
{

mat4 mvm = CAM_MODEL * CAM_VIEW;
mvm.Transpose();
vec3 dirX, dirY;
	dirX.x = mvm.m[0];
	dirX.y = mvm.m[4];
	dirX.z = mvm.m[8];

	dirY.x =	mvm.m[1];
	dirY.y =	mvm.m[5];
	dirY.z =	mvm.m[9];


	float a = fov / 2.0;
float cotangent = 1.0 / tan( a * imopi );

float ax = z_near / cotangent;

float screen_w = 2.0*ax;

float screen_h = screen_w;// * yratio;

screen_w = screen_w * aspect;

float scr_coord_x = float(x) / float(sw);
float scr_coord_y = float(sh - y) / float(sh);


vec3 dir = FPP_CAM->ReturnFrontVector();

//move to lower left corner
vec3 start_pos = (FPP_CAM->pos + dir * z_near) + (-dirX * (screen_w / 2.0)) + (-dirY * (screen_h/2.0));

vec3 start = start_pos + (dirX * (screen_w * scr_coord_x)) + (dirY * (screen_h * scr_coord_y));

return Normalize( vectorAB(FPP_CAM->pos, start) );
}

FPP_CAM->pos camera position

fov is fovy actually, so it has to be converted into horizontal fov

since its opengl code int y has to be scrheight - y

CAM_MODEL is always identity

z_far is not used at all

Master & Mentor
https://sites.google.com/site/customprog/

dave

2,188

February 21, 2016 12:55 PM

You might also consider using a pick buffer which is a render target where each pixel contains an ID that is the object that covers that pixel. Therefore sampling the pick buffer under the cursor gives you the object the mouse is over. This doesn't solve box selection though, but is a simple way of doing it that doesnt involve alot of projection/unprojection. Note also that its particularly good for picking terrain, where typically you don't want to be doing ray triangle intersection.

Correct way of doing Mouse Picking

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Correct way of doing Mouse Picking

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Reticulating splines