# Ray Picking doesn't doesn't work with CFirstPersonCamera

## Recommended Posts

firegod666    122
Hi all, Below is my ray picking mesh code that only works if I use the model camera. I need it to work with a first person camera but when I use CFirstPersonCamera it hits all the time even when looking at a blank meshless screen. I'm using the latest DX9 SDK where all the sample code uses the DXUT... code in a common directory. Also you'll notice I lock the vertex buffers etc, is this really needed for D3DXIntersect? Please excuse the code as it's based on the MS sample pick, and I've been bodging it around for 2 days now trying to get it to work. I'll optimise and clean it up and repost it here when some kind soul helps me fix it. Regards FG.
FLAG xraypick(OBJECT3D_ID oid,INT32 x, INT32 y)
{
OBJECT3D	*optr;							/* pointer to the current object in list */
D3DXVECTOR3 vPickRayDir;
D3DXVECTOR3 vPickRayOrig;
IDirect3DDevice9* pD3Device = DXUTGetD3DDevice();
const D3DSURFACE_DESC* pd3dsdBackBuffer = DXUTGetBackBufferSurfaceDesc();

optr = &all_objects[oid];

g_dwNumIntersections = 0L;

const D3DXMATRIX *pmatProj = g_Camera.GetProjMatrix();           /* Get the Pick ray from the mouse position */

D3DXVECTOR3 v;                                                   /* Compute the vector of the Pick ray in screen space */
v.x =  ( ( ( 2.0f * cursor.x ) / pd3dsdBackBuffer->Width  ) - 1 ) / pmatProj->_11;
v.y = -( ( ( 2.0f * cursor.y ) / pd3dsdBackBuffer->Height ) - 1 ) / pmatProj->_22;
v.z =  1.0f;

const D3DXMATRIX matView = *g_Camera.GetViewMatrix();            /* Get the inverse view matrix*/
const D3DXMATRIX matWorld = *g_Camera.GetWorldMatrix();
D3DXMATRIX mWorldView;
matrix_mul((MATRIX *)&mWorldView,(MATRIX *)&matWorld,(MATRIX *)&matView);
D3DXMATRIX m;
D3DXMatrixInverse( &m, NULL, &mWorldView );

vPickRayDir.x  = v.x*m._11 + v.y*m._21 + v.z*m._31;              /* Transform the screen space Pick ray into 3D space */
vPickRayDir.y  = v.x*m._12 + v.y*m._22 + v.z*m._32;
vPickRayDir.z  = v.x*m._13 + v.y*m._23 + v.z*m._33;
vPickRayOrig.x = m._41;
vPickRayOrig.y = m._42;
vPickRayOrig.z = m._43;

LPDIRECT3DVERTEXBUFFER9 pVB;
LPDIRECT3DINDEXBUFFER9  pIB;

optr->mesh->GetMesh()->GetVertexBuffer( &pVB );
optr->mesh->GetMesh()->GetIndexBuffer( &pIB );

WORD*      pIndices;
D3DVERTEX* pVertices;

pIB->Lock( 0, 0, (void**)&pIndices, 0 );
pVB->Lock( 0, 0, (void**)&pVertices, 0 );

BOOL bHit;                                                         /* Collect only the closest intersection */
DWORD dwFace;
FLOAT fBary1, fBary2, fDist;
if(D3DXIntersect(optr->mesh->GetMesh(),&vPickRayOrig,&vPickRayDir,&bHit,&dwFace,&fBary1,&fBary2,&fDist,NULL,NULL) != D3D_OK)
return(0);
if( bHit )
{
g_dwNumIntersections = 1;
g_IntersectionArray[0].dwFace = dwFace;
g_IntersectionArray[0].fBary1 = fBary1;
g_IntersectionArray[0].fBary2 = fBary2;
g_IntersectionArray[0].fDist = fDist;
}
else
{
g_dwNumIntersections = 0;
}

pVB->Unlock();
pIB->Unlock();
SAFE_RELEASE(pVB);
SAFE_RELEASE(pIB);

return(bHit);
}



##### Share on other sites
ankhd    2304
hey there been there same problem, none of the tut's work this is how they hold every one back the dogs.
anyway here is what you need to do it worked for me but It dosen't work for the
D3DXIntersectTri function or I have some thing wrong some were.

here is what your looking for use this
D3DXVECTOR3 *rt = NULL;//returned from call	D3DXVECTOR3 v, vmouse, temp;	vmouse.x = ptCursor.x;// + GetSystemMetrics(SM_CXSIZEFRAME);	vmouse.y = ptCursor.y;	vmouse.z = 0;//ViewPort->MinZ;//0.001f;//campos->z;			//Call D3DXVec3Unproject two times, once with MouseX,MouseY,0 and MouseX,MouseY,1.	//This returns you two points in 3d space which form a ray. Use that ray to do intersection tests.	rt = D3DXVec3Unproject(&rmc->m_rayDirection,//rayDir,//D3DXVECTOR3 *pOut,							&vmouse,//CONST D3DXVECTOR3 *pV,							ViewPort,//CONST D3DVIEWPORT9 *pViewport,							matProj,//CONST D3DXMATRIX *pProjection,							matView,//CONST D3DXMATRIX *pView,							pWorld);//CONST D3DXMATRIX *pWorld	if(rt == NULL)	{		MessageBeep(MB_OK);		return S_OK;	}		//temp.x = campos->x;//ptCursor.x;	//temp.y = campos->y;//ptCursor.y;	//vmouse = temp;// - (*campos);	//vmouse.z = campos->z;		vmouse.z = 1;//ViewPort->MaxZ;//0.99f;	//D3DXMATRIX matViewInv;		//D3DXMatrixInverse(&matViewInv, NULL, matView );	rt = D3DXVec3Unproject(&rmc->m_rayOrigin,//rayOrigin,//D3DXVECTOR3 *pOut,							&vmouse,//CONST D3DXVECTOR3 *pV,							ViewPort,//CONST D3DVIEWPORT9 *pViewport,						matProj,//CONST D3DXMATRIX *pProjection,						matView,//CONST D3DXMATRIX *pView,						pWorld);//CONST D3DXMATRIX *pWorld	if(rt == NULL)	{		MessageBeep(MB_OK);		return S_OK;//error	}	rmc->m_rayDirection = rmc->m_rayDirection - rmc->m_rayOrigin;	rayDir		= rmc->m_rayDirection;;	rayOrigin	= rmc->m_rayOrigin;

##### Share on other sites
firegod666    122
Quote:
 Original post by ankhdhey there been there same problem, none of the tut's work this is how they hold every one back the dogs.anyway here is what you need to do it worked for me but It dosen't work for the D3DXIntersectTri function or I have some thing wrong some were.here is what your looking for use this*** Source Snippet Removed ***

Thanks for that. I tried that method using unproject but I was using D3DXIntersect so maybe thats why it didn't work. What triangle intersect code do you use with that example?

##### Share on other sites
firegod666    122
Bump as I still need an solution.

##### Share on other sites
ankhd    2304
Hey that code does work if its not working you need to check the world matrix
that will also stop it from working as I found out. When I set the world for the object I used a scale and I set the z to zero and that was the problem so check all you matrices. and post your new code so we can see whats wrong.

here his how I get my view matrix from the camera
void Camera2::getViewMatrix(D3DXMATRIX* V){	D3DXMatrixIdentity(V);	// Keep camera's axes orthogonal to eachother	D3DXVec3Normalize(&_look, &_look);	D3DXVec3Cross(&_up, &_look, &_right);	D3DXVec3Normalize(&_up, &_up);	D3DXVec3Cross(&_right, &_up, &_look);	D3DXVec3Normalize(&_right, &_right);	// Build the view matrix:	float x = -D3DXVec3Dot(&_right, &_pos);	float y = -D3DXVec3Dot(&_up, &_pos);	float z = -D3DXVec3Dot(&_look, &_pos);	(*V)(0,0) = _right.x; (*V)(0, 1) = _up.x; (*V)(0, 2) = _look.x; (*V)(0, 3) = 0.0f;	(*V)(1,0) = _right.y; (*V)(1, 1) = _up.y; (*V)(1, 2) = _look.y; (*V)(1, 3) = 0.0f;	(*V)(2,0) = _right.z; (*V)(2, 1) = _up.z; (*V)(2, 2) = _look.z; (*V)(2, 3) = 0.0f;	(*V)(3,0) = x;        (*V)(3, 1) = y;     (*V)(3, 2) = z;       (*V)(3, 3) = 1.0f;}