Picking Problem - Pictures inside

Started by
4 comments, last by Bimble Bob 18 years ago
Hello all! I finally managed to get my Tile picking code working well badly anyway. As the picture below illustrates something is seriously wrong here. I was following the guidelines on http://www.toymaker.info/Games/html/picking.html (Which is really good BTW) and here is my MouseOver() function which detects if the mouse is over a tile:

bool TILE::MouseOver()
{
	
	m3d.x = (((2.0f * mousex) / width) - 1) / ProjMat._11; //Converts 2D mouse coordinates into 3D world coordinates
	m3d.y = -(((2.0f * mousey) / height) - 1) / ProjMat._22; //Converts 2D mouse coordinates into 3D world coordinates
	m3d.z = 1.0f;

	//Get the Inverse of the view matrix
	D3DXMatrixInverse(&invView, NULL, &ViewMat);

	//Turn the screen "pick" ray into 3D coordinates
	rayDir.x = m3d.x * invView._11 + m3d.y * invView._21 + m3d.z * invView._31;
	rayDir.y = m3d.x * invView._12 + m3d.y * invView._22 + m3d.z * invView._32;
	rayDir.z = m3d.x * invView._13 + m3d.y * invView._23 + m3d.z * invView._33;
	rayBegin.x = invView._41;
	rayBegin.y = invView._42;
	rayBegin.z = invView._43;
	
	if(D3DXIntersectTri(&W1, &W2, &W3, &rayBegin, &rayDir, &u, &v, &dist) == TRUE)
	{
		return TRUE;
	}
	else
	{
		return FALSE;
	}

}

And here is a picture of my problems: Picking Problems! Please help me out [help] so close but yet so far!
It's not a bug... it's a feature!
Advertisement
Hey Dom, you seem to transform the ray implied by the mouse, only by the inverse view matrix. This would leave both the ray and its origin in world coordinates. So, the question is, are the tile vertices that you are passing to D3DXIntersect also in global coordinates? It seems to me that they're probably not, and that's why you get bogus picking.

If you only have access to the untransformed, local space coordinates of the tiles, you will also need to find the ray and origin in the model's local space.
To achieve that, replace the matrix invView below:
rayDir.x = m3d.x * invView._11 + m3d.y * invView._21 + m3d.z * invView._31;	rayDir.y = m3d.x * invView._12 + m3d.y * invView._22 + m3d.z * invView._32;	rayDir.z = m3d.x * invView._13 + m3d.y * invView._23 + m3d.z * invView._33;	rayBegin.x = invView._41;	rayBegin.y = invView._42;	rayBegin.z = invView._43;

with a matrix m, such that m = InverseView*InverseWorldMatrix, where WorldMatrix is the world transformation you used to render the tiles.
No the vertices for the triangles are transformed into World View when they are created and then stored in vectors which can be accessed later. here is how I do it:
D3DXVec3TransformCoord(&W1, &D3DXVECTOR3(-1.0f, -1.0f, 0.0f), &WorldMat);	D3DXVec3TransformCoord(&W2, &D3DXVECTOR3(-1.0f, 0.5f, 0.0f), &WorldMat);	D3DXVec3TransformCoord(&W3, &D3DXVECTOR3(0.0f, -1.0f, 0.0f), &WorldMat);	D3DXVec3TransformCoord(&W4, &D3DXVECTOR3(0.0f, 0.5f, 0.0f), &WorldMat);


It may be a problem here I'm not sure but after researching object picking it seems there are two ways of casting a ray from the cursor...

Also in regards to your suggestion:

D3DXMatrixInverse(&invView, NULL, &ViewMat);
D3DXMatrixInverse(&invWorld, NULL, &WorldMat);

D3DXMATRIX m = (invView * invWorld);

Then what do we do with the Matrix m, should that be passed to InteresectTri()?

It's not a bug... it's a feature!
This is weird... It seems that you *are* feeding it vertices in world coordinates...

Quote:
Then what do we do with the Matrix m, should that be passed to InteresectTri()?

No, you use the matrix "m", instead of "invView", in the part of your snippet where you calculate rayDir and rayBegin.

I don't know... I was almost certain that this was the problem. If you want to try it -just in case-, you'll have to pass the original un-transformed vertices to D3DXIntersect:
D3DXVECTOR3(-1.0f, -1.0f, 0.0f), D3DXVECTOR3(-1.0f, .5f, 0.0f) etc.

edit:
Quote:
It may be a problem here I'm not sure but after researching object picking it seems there are two ways of casting a ray from the cursor...

If you mean the two implementations, one with invView, and the other with invView*invWorld, they're essentially the same thing. It only gets down to the frame of reference your vertices are expressed in.
Yeah ill try using the untransformed vertices, I'll report back with my findings.
No, passing the untrasformed vertices to D3DXIntersectTri()doesn't do anything not even bad picking.
It's not a bug... it's a feature!
Woo I've made some progress I changed:

m3d.x = (((2.0f * mousex) / width) - 1) / ProjMat._11; //Converts 2D mouse coordinates into 3D world coordinates	m3d.y = -(((2.0f * mousey) / height) - 1) / ProjMat._22; //Converts 2D mouse coordinates into 3D world coordinates


to

m3d.x = (((2.0f * mousex) / d3dpp.BackBufferWidth) - 1) / ProjMat._11; //Converts 2D mouse coordinates into 3D world coordinates	m3d.y = -(((2.0f * mousey) / d3dpp.BackBufferHeight) - 1) / ProjMat._22; //Converts 2D mouse coordinates into 3D world coordinates


And now I can select the tiles in the first column perfectly however the rest are still totally off! Oh I should also say the intersect is perfect now as far as no tile is selected if the cursor is over the black. If that made any sense.

I think it might be a problem with the way I am creating and checking for intersect on my tiles. i am using two for loops to create the tile grid and this may be messing up the Tile ID numbers... I'll experiment.

Hehe yeah it was that it was creating the tiles in a totally weird order so tile number 2 might've been next to tile number 37! But yeah it all works now but the framerate has dropped DRAMATICALLY! I am checking for mouse over every frame, could that be why? And also I am rendering and checking all the tiles even if they are not on screen, how do you check if an object isn't on screen?
It's not a bug... it's a feature!

This topic is closed to new replies.

Advertisement