I am struggling with the math to begin with and reading multiple papers on how to pick a triangle from a 2d mouse position and I somewhat understand the high level plan. When I look at the pick sample though I don''t understand the math and can''t find some of the variables used. I always found the framework to be confusing. To keep this post simple I''ll just ask my first question given this snippet from the sample: // Compute the vector of the pick ray in screen space D3DXVECTOR3 v; v.x =(((2.0f * ptCursor.x) / m_d3dsdBackBuffer.Width) - 1) / matProj._11; v.y= -(((2.0f * ptCursor.y) / m_d3dsdBackBuffer.Height) - 1) / matProj._22; v.z = 1.0f; can someone explain the math? matproj_11 and _22 are normally multiplied by a vertex so I understand we are undoing that by dividing for the x and y, I think, but I dont understand the rest of the line why z=1 instead of -1? in a lh system - is towards the user and + is into the screen right? I guess I don''t know if we are doing origin of the ray or direction? how can i retrieve backbuffer width and height when I didn''t use the framework in my application? (I dont know if it''s needed because I don''t understand the math here). All I do understand is that I must somehow convert 2d mouse coords into worldspace coordinates by taking a ray from the near clip plane and that inversing the matrix transformations is needed. This is getting way too complicated for me. I read the article on the mvp site and the pick sample + searched some forumns and newsgroups, but none explain the math very well at all, at least not on my level. Thanx, Christopher

i''m going to just go through everything i can, making no assumptions. so please don''t take offense if some of this is obvious:

ptCursor.x and .y define the cursor position within the rendering window. this window has its own coordinate system which has an origin at the upper-left of the window''s client area and extends left and down. units of measure are in pixels.

m_d3dsBackBuffer.Width and .Height are used here as the rendering window''s extents (dimensions.) if you don''t have access to the back buffer''s dimensions then you can use a GetClientRect call.

2*ptCursor.x/m_d3dsBackBuffer.Width and 2*ptCursor.y/m_d3dsBackBuffer.Height convert the cursor position''s window coordinates to values between 0 and 2, inclusive.

subtracting 1 from the above scaled values converts the cursor''s window coordinates to values between -1 and 1, inclusive.

the result of the above steps is to convert the cursor position to a new coordinate system which has an origin in the middle of the screen and axes that extend one unit length outward from that center point. negating the y value flips the y axis so that negative y points down and positive y points up.

dividing the resulting x and y values given above, by matProj._11 and matProj._22 respectively, scales the cursor position to the size of the viewport as seen in the camera''s coordinate system. setting the z value to 1 is done to translate the position to the back of the view frustum (which extends from 0 to 1 in the camera coordinate system.) so what you end up with is a point, defined within the camera coordinate system, on the back of the view frustum.

running the above camera coordinate system point through the inverse of the view matrix will then convert the point to a normalized vector in the world coordinate system. you can think of the vector as originating at the camera position, passing through the viewport at the cursor position, and terminating at the transformed cursor position on the back of the view frustum.

i hope this is clear enough now. the easiest way to think of this is that all of these steps are simply undoing transformations that would be done if the pick point were originally given in the world coordinate system.

Thank you so much! That is the best information I have seen
on this to date. And believe me, I been looking

I also read that this only works if the world matrix
is an identity matrix, but there is some stuff I need
to read and I need to code/comment what you have said
so as to absorb it before i get into that. I am
predicting I just multiply the vector by the inverse
of the world matrix.

Thanx,
Christopher

but what happens when the mesh is rotated or translated..

i tried to move my mesh.. and it no longer picks properly..

You have to multiply your ray by the inverse of the matrix you used to transform your mesh in the first place.

Use D3DXVec3TransformCoord to transform the origin of your ray, and use D3DXVec3TransformNormal to transform the direction of your ray.

neneboricua

hi,

Took me a while, too. Microsoft uses the inverse view matrix

here''s some code (ptClient is the mouse coord within window client area, winsize->x and winsize->y
are the width and height of the window, pmesh if a pointer to a custom mesh wrapper i use)

// calculate inverse of World*View*Project
D3DXMATRIX mxInverseWVP;
D3DXMatrixMultiply( &mxInverseWVP, &mxView, &mxProj );
D3DXMatrixMultiply( &mxInverseWVP, &matWorld, &mxInverseWVP );
D3DXMatrixInverse( &mxInverseWVP, NULL, &mxInverseWVP );

// find the pick ray, which is (x, y, 0) to (x, y, 1), where x and y are the mouse coords, scaled to -1, 1 and 1, -1
float x = (float)ptClient->x/(float)winsize->x * 2.0f - 1.0f;
float y = (float)ptClient->y/(float)winsize->y * -2.0f + 1.0f;
D3DXVECTOR3 vRayStart(x, y, 0.0f);
D3DXVECTOR3 vRayEnd(x, y, 1.0f);
D3DXVec3TransformCoord(&vRayStart, &vRayStart, &mxInverseWVP);
D3DXVec3TransformCoord(&vRayEnd, &vRayEnd, &mxInverseWVP);
vRayEnd -= vRayStart;

// check mesh for intersection
BOOL bHit;
DWORD dwFace;
FLOAT fBary1, fBary2, fDist;
D3DXIntersect( m_pMesh, &vRayStart, &vRayDir, &bHit, &dwFace, &fBary1, &fBary2, &fDist, NULL, NULL);
return bHit;

Stijn

oops, vRayDir should be vRayEnd in the last line of code -- sorry

Stijn