I know this question has been asked a zillion times before, but I just can't get this to work. I followed
this tutorial to get the ray, and then I use d3dxintersect to see if the ray collides with a mesh. I don't know if where I'm messing up, so any ideas would be nice.
//first, some constants
#define FOV D3DX_PI/4.0f
#define ASPECT 1.0f
#define NEAR 1.0f
#define FAR 100.0f
//later, when I create the projection matrix
D3DXMatrixPerspectiveFovLH( &matProj, FOV, ASPECT, NEAR, FAR);
D3DDevice->SetTransform( D3DTS_PROJECTION, &matProj );
//and this code is in the msgproc() function, under the WM_MOUSEMOVE case
D3DXVECTOR3 vecstart,vec;
ScreenTo3D(LOWORD(lParam),HIWORD(lParam),&vecstart,&vec); //get the ray based on the mouse position
//prepare for d3dxintersect()
BOOL hit = false;
DWORD fi;
float u,v,dist;
const D3DXVECTOR3 dir = vec;
const D3DXVECTOR3 sp = vecstart;
D3DXIntersect(SomeMesh,&sp,&dir,&hit,&fi,&u,&v,&dist,NULL,NULL);
if(hit)
{
MessageBox(NULL,"Mouse over mesh","blah",NULL);
}
And here's the ScreenTo3D function
void ScreenTo3D(int mx,int my,D3DXVECTOR3 * p1,D3DXVECTOR3 * p2)
{
int dy,dx;
dx = tanf(FOV * 0.5f) * (mx / (GameWidth / 2) - 1.0f) / ASPECT;
dy = tanf(FOV * 0.5f) * (1.0f - my / (GameHeight / 2));
//calculate the beginning and end points of the ray
p1->x = dx * NEAR;p1->y = dy * NEAR;p1->z = NEAR;
p2->x = dx * FAR;p2->y = dy * FAR;p2->z = FAR;
//inverse them by the view matrix
D3DXMATRIX invMat; //the inverse of the view matrix
D3DXMatrixInverse(&invMat,NULL,&matView); //inverse the view mat
D3DXVec3TransformCoord(p1,p1,&invMat);
D3DXVec3TransformCoord(p2,p2,&invMat);
//inverse them by the world matrix
D3DXMATRIX invwMat; //the inverse of the view matrix
D3DXMatrixInverse(&invwMat,NULL,&matWorld); //inverse the view mat
D3DXVec3TransformCoord(p2,p2,&invwMat);
D3DXVec3TransformCoord(p2,p2,&invwMat);
}
NOTE: matProj, matView, and matWorld are defined as global variables.
So, where am I messing up?
[Edited by - jmutch on October 1, 2004 11:40:13 PM]
