Sign in to follow this  
jmutch

ray casting and d3dxintersect() *solved*

Recommended Posts

jmutch    397
I know this question has been asked a zillion times before, but I just can't get this to work. I followed this tutorial to get the ray, and then I use d3dxintersect to see if the ray collides with a mesh. I don't know if where I'm messing up, so any ideas would be nice.
//first, some constants
#define FOV D3DX_PI/4.0f
#define ASPECT 1.0f
#define NEAR 1.0f
#define FAR 100.0f

//later, when I create the projection matrix
D3DXMatrixPerspectiveFovLH( &matProj, FOV, ASPECT, NEAR, FAR);
D3DDevice->SetTransform( D3DTS_PROJECTION, &matProj );

//and this code is in the msgproc() function, under the WM_MOUSEMOVE case

D3DXVECTOR3 vecstart,vec;
ScreenTo3D(LOWORD(lParam),HIWORD(lParam),&vecstart,&vec); //get the ray based on the mouse position

//prepare for d3dxintersect()
BOOL hit = false;
DWORD fi;
float u,v,dist;
const D3DXVECTOR3 dir = vec;
const D3DXVECTOR3 sp = vecstart;

D3DXIntersect(SomeMesh,&sp,&dir,&hit,&fi,&u,&v,&dist,NULL,NULL);
if(hit)
{
	MessageBox(NULL,"Mouse over mesh","blah",NULL);
}



And here's the ScreenTo3D function
void ScreenTo3D(int mx,int my,D3DXVECTOR3 * p1,D3DXVECTOR3 * p2)
{
	int dy,dx;

	dx = tanf(FOV * 0.5f) * (mx / (GameWidth / 2) - 1.0f) / ASPECT;
	dy = tanf(FOV * 0.5f) * (1.0f - my / (GameHeight / 2));

	//calculate the beginning and end points of the ray
	p1->x = dx * NEAR;p1->y = dy * NEAR;p1->z = NEAR;
	p2->x = dx * FAR;p2->y = dy * FAR;p2->z = FAR;


	//inverse them by the view matrix
	D3DXMATRIX invMat; //the inverse of the view matrix
	D3DXMatrixInverse(&invMat,NULL,&matView); //inverse the view mat

	D3DXVec3TransformCoord(p1,p1,&invMat);
	D3DXVec3TransformCoord(p2,p2,&invMat);


	//inverse them by the world matrix
	D3DXMATRIX invwMat; //the inverse of the view matrix
	D3DXMatrixInverse(&invwMat,NULL,&matWorld); //inverse the view mat
	
	D3DXVec3TransformCoord(p2,p2,&invwMat);
	D3DXVec3TransformCoord(p2,p2,&invwMat);


}



NOTE: matProj, matView, and matWorld are defined as global variables. So, where am I messing up? [Edited by - jmutch on October 1, 2004 11:40:13 PM]

Share this post


Link to post
Share on other sites
jmutch    397
Ok, I'm an idiot. I had,

int dy,dx;

When I should have declared that as a float. But it still doesn't work. [crying] Any ideas?


PS,
I changed this

dx = tanf(FOV * 0.5f) * (mx / (GameWidth / 2) - 1.0f) / ASPECT;
dy = tanf(FOV * 0.5f) * (1.0f - my / (GameHeight / 2));

to this

float HalfGameWidth = GameWidth / 2.0f;
float HalfGameHeight = GameHeight / 2.0f;

float tang = tanf((float)FOV * 0.5f);
dx = tang * (mx / HalfGameWidth - 1.0f) / (float)ASPECT;
dy = tang * (1.0f - my / HalfGameHeight);

Share this post


Link to post
Share on other sites
jmutch    397
Does any one know of a demo somewhere that does this? It seems that the pick sample in the 9.0c sdk (I remember it being there in 9.0) dissapeared for me.

Share this post


Link to post
Share on other sites
jmutch    397
K, I solved it. The final ScreenTo3D function looks like this. (thought I'd put it up here as a resource)


void ScreenTo3D(int mx,int my,D3DXVECTOR3 * p1,D3DXVECTOR3 * p2)
{
float dy,dx;
float HalfGameWidth = GameWidth / 2.0f;
float HalfGameHeight = GameHeight / 2.0f;
float tang = tanf((float)FOV * 0.5f);


dx = tang * (mx / HalfGameWidth - 1.0f) / (float)ASPECT;
dy = tang * (1.0f - my / HalfGameHeight);

//calculate the beginning and end points of the ray
p1->x = dx * NEAR;p1->y = dy * NEAR;p1->z = NEAR;
p2->x = dx * FAR;p2->y = dy * FAR;p2->z = FAR;

//inverse them by the view matrix
D3DXMATRIX invMat; //the inverse of the view matrix
D3DXMatrixInverse(&invMat,NULL,&matView); //inverse the view mat

D3DXVec3TransformCoord(p2,p2,&invMat);
D3DXVec3TransformCoord(p1,p1,&invMat);

//inverse them by the world matrix
D3DXMATRIX invwMat; //the inverse of the view matrix
D3DXMatrixInverse(&invwMat,NULL,&matWorld); //inverse the view mat

D3DXVec3TransformCoord(p2,p2,&invwMat);
D3DXVec3TransformCoord(p1,p1,&invwMat);

D3DXVec3Normalize(p2,&(*p2 - *p1));

}




Thanks so much guys, I never could have done it without you [grin]

Share this post


Link to post
Share on other sites
jmutch    397
Quote:
Original post by OrenGL
How about D3DXVec3Unproject


Now you tell me, not one hour ago when I was about ready to pull out my hair. *sobs*

Man, that looks soooo much easier.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this