Screen to World Ray

Started by
28 comments, last by GuyWithBeard 11 years, 4 months ago
I'm using btBvhTriangleMeshShape for the terrain and btConvexHullShape for some other meshes, one important question, what physics shape should I use for buildings, airstrikes, vehicles and other interiors such as tables, chairs, etc? Remember when the player shoot I have to place a bullet hole billboard on exactly the same point the player is shooting at and since raytest has nothing to do with rendering and it's working according to bullet physics, I have to choose the right shapes for loading physics meshes so when the bullet hole is placed, its position should be realistic.

I'm drawing the line and I see that I'm getting somewhere close to doing the right thing, here is my code:
inline D3DXVECTOR3 GetPickRay(float x, float y)
{
D3DXMATRIX m_matProj = camera->projectionMatrix();
D3DXMATRIX m_matView = camera->viewMatrix();


D3DXVECTOR3 coord;
coord.x = ( ( ( 2.0f * x ) / WND_WIDTH ) - 1 );
coord.y = -( ( ( 2.0f * y ) / WND_HEIGHT ) - 1 );
coord.z = 1.0f;
// Back project the ray from screen to the far clip plane
coord.x /= m_matProj._11;
coord.y /= m_matProj._22;
D3DXMATRIX matinv;
D3DXMatrixInverse(&matinv, NULL, &m_matView);
coord*=1000000.0f;
D3DXVec3TransformCoord(&coord, &coord, &matinv);
return coord;
}

POINT m_cursorCoords;
GetCursorPos(&m_cursorCoords);
D3DXVECTOR3 m_rayFrom = camera->GetPosition();
D3DXVECTOR3 m_rayTo = GetPickRay(WND_WIDTH / 2, WND_HEIGHT / 2);
btVector3 btRayFrom = btVector3(m_rayFrom.x, m_rayFrom.y, m_rayFrom.z);
btVector3 btRayTo = btVector3(m_rayTo.x, m_rayTo.y, m_rayTo.z);

if (MouseLeftButtonPressed())
{
DWORD dwHit = BuletRaytest(m_rayFrom, m_rayTo);

// dwHit should store the mesh that got shoot, create bullet hole billboard and apply damage here...
}



The above code is not shooting from the center of the screen, it's somehow close.
Advertisement
Well, there are a few problems with your approach. First of all it's a lot heavier to physically simulate convex hull shapes than, say, boxes. A lot of objects can be simplified vastly on the physics side. For example, characters are often simulated as capsules until they, for one reason or another become rag dolls.

Another problem is that btConvexHullShape can only represent convex shapes, which means that any concave geometry you may have will be wrong anyway.

Things like this make using the physics engine for placing decals (bullet holes etc.) a bad choice. It's usually better to perform the ray casting on the graphics layer. If you update your D3DX stuff to the new DirectXMath library (which is basically the new version of D3DX) you get a lot of collision detection functionality for free. What was once called xnacollision is now included in the core lib and provides, among other things, ray to triangle intersection which is probably what you want for placing decals.
Do you mean that I should use D3DXIntersect() to make the player shooting instead of using Bullet Physics Raytest? I believe that bullet raytest is used in picking/weapon shooting/etc...
D3DXIntersect is part of D3DX. I suggested that you replace D3DX with the newer and more powerful version, DirectXMath.

You can of course use bullet to perform the ray casts, but remember that there might be situations where you cannot (easily) represent the visual objects using the physics meshes. Bullet (or any other physics engine) is useful for ray casting when you want to know if a bullet shot from a weapon has hit an object, but it's less useful when you want the exact position on a surface of a mesh, for placing a bullet hole decal. Of course, all this depends on how your physics meshes, and indeed all your meshes, are constructed. Some games might very well use the same geometry for both physics and visuals.

http://www.esenthel.com/wiki/index.php?title=Physical_Body

Check out the images in the article above. As you see the physical meshes are often not the same as the visual ones. For example casting a ray onto the capsule around the character and placing a bullet hit billboard at the hit location would make the billboard float in the air, instead of being drawn on the character.

And as you probably know, the character in the image above cannot be represented with a btConvexHullShape.
According to:
http://msdn.microsof...3(v=vs.85).aspx

The DirectXMath library is designed for C++ developers working on games and DirectX graphics in Windows Store apps and traditional desktop apps for Windows 8 and later.

I want the game to work on earlier versions of Windows such as Windows XP and Windows 7, so I guess DirectXMath will not be the choice.

Here is the code that I'm using:

// When the user press the left mouse button, this code will be executed on EVERY mesh
// matWorld represent the current mesh world matrix
// Shooting
// get the current transform matrices
D3DXMATRIX matProjection, matView, matWorld, matInverse;
m_d3dDevice->GetTransform(D3DTS_PROJECTION, &matProjection);
m_d3dDevice->GetTransform(D3DTS_VIEW, &matView);
m_d3dDevice->GetTransform(D3DTS_WORLD, &matWorld);
// use the mouse coordinates to get the mouse angle
POINT MousePos;
GetCursorPos(&MousePos);
float xAngle = (((2.0f * MousePos.x) / WND_WIDTH) - 1.0f) / matProjection(0, 0);
float yAngle = (((-2.0f * MousePos.y) / WND_HEIGHT) + 1.0f) / matProjection(1, 1);
D3DXVECTOR3 origin, direction;
origin = D3DXVECTOR3(0.0f, 0.0f, 0.0f);
direction = D3DXVECTOR3(xAngle, yAngle, 1.0f);
// find the inverse matrix
D3DXMatrixInverse(&matInverse, NULL, &(matWorld * matView));
// convert origin and direction into model space
D3DXVec3TransformCoord(&origin, &origin, &matInverse);
D3DXVec3TransformNormal(&direction, &direction, &matInverse);
D3DXVec3Normalize(&direction, &direction);

DWORD fi;
FLOAT u, v;
FLOAT dist;
BOOL hit;
D3DXIntersect(staticMesh,&origin,&direction,&hit,&fi,&u,&v,&dist,NULL,NULL);
if(hit)
{
stringstream ss;
ss << "Mouse over: " << currentMeshId;
SetWindowText(GetForegroundWindow(), ss.str().c_str());
}



Tried the above code but it's not working exactly as expected, what's wrong with that code?

I want the game to work on earlier versions of Windows such as Windows XP and Windows 7, so I guess DirectXMath will not be the choice.


Actually it will. You can use the earlier version of the lib, called XNAMath, which is compatible with earlier versions of windows. That version lies between D3DX and DirectXMath, but it's closer to DirectXMath than D3DX. Also it has nothing to do with XNA, despite its name. Confusing I know. Read more about it here if you are interested:

http://blogs.msdn.com/b/chuckw/archive/2012/03/27/introducing-directxmath.aspx

XNAMath has a library called xnacollision which does what you need.

I don't see anything wrong with the code. I have never seen any code take the address of an unnamed variable like you do on line 18, but it might work so that might not be the problem. I think some compilers might go haywire because of that, but I am too lazy to test that out right now. I would probably assign to a local variable before taking the address.

However, if you are still trying to cast a ray through the center of the screen, why don't you comment out the angle calculations for now and just use zeroes as I and others have already suggested? That way you can bypass a lot of math, and see if your casting works. To recap, the direction would just be (0, 0, 1) in view space for a ray straight forward from the camera.
I found that it's working :) one problem I notice, lets say that I have 10 characters standing in one row, in the above code one shot will intersect the 10 characters which is a problem, since one shot should not hit 10 characters.

I want the ray to intersect the closest mesh only, then I will apply damage to the closest mesh.

I really appreciate your help.
After working on it, I came out with the following code:
DWORD closestDistance = 0;
DWORD currentMeshId = 0;
bool finalHit = false;
for(DWORD i = 0; i < meshes.size(); i++)
{
// When the user press the left mouse button, this code will be executed on EVERY mesh
// matWorld represent the current mesh world matrix
// Shooting
// get the current transform matrices
D3DXMATRIX matProjection, matView, matWorld, matInverse;
d3ddev->GetTransform(D3DTS_PROJECTION, &matProjection);
d3ddev->GetTransform(D3DTS_VIEW, &matView);
matWorld = meshes->GetWorldMatrix();
// use the mouse coordinates to get the mouse angle
POINT MousePos;
GetCursorPos(&MousePos);
float xAngle = (((2.0f * MousePos.x) / WND_WIDTH) - 1.0f) / matProjection(0, 0);
float yAngle = (((-2.0f * MousePos.y) / WND_HEIGHT) + 1.0f) / matProjection(1, 1);
D3DXVECTOR3 origin, direction;
origin = D3DXVECTOR3(0.0f, 0.0f, 0.0f);
direction = D3DXVECTOR3(xAngle, yAngle, 1.0f);
// find the inverse matrix
D3DXMatrixInverse(&matInverse, NULL, &(matWorld * matView));
// convert origin and direction into model space
D3DXVec3TransformCoord(&origin, &origin, &matInverse);
D3DXVec3TransformNormal(&direction, &direction, &matInverse);
D3DXVec3Normalize(&direction, &direction);

DWORD fi;
FLOAT u, v;
FLOAT dist;
BOOL hit;
D3DXIntersect(meshes->GetMesh(), &origin, &direction, &hit, &fi, &u, &v, &dist, NULL, NULL);
if(hit)
{
if (dist < closestDistance || closestDistance == 0)
{
currentMeshId = meshes->meshId;
closestDistance = dist;
finalHit = true;
}
}
}

if (finalHit)
{
// -- Closest hit
stringstream ss;
ss << "Mouse over: " << currentMeshId;
SetWindowText(GetForegroundWindow(), ss.str().c_str());
} else {
SetWindowText(GetForegroundWindow(), "No hit!");
}


I guess the code is working fine to pick the closest mesh, however I'm not sure if it will reduce the FPS or could cause the game to lag if I have thousands of meshes (which is expected in a medium/large mission.

I want the ray to intersect the closest mesh only, then I will apply damage to the closest mesh.


Glad it is working, you should read more about D3DXIntersect() and its last two paramters. But the hit and dist parameters are for the first/closest hit.
I guess the problem here is that when using a function such as D3DXIntersect you need to know which mesh to test, ie. the first parameter to the function. Testing all meshes can indeed be slow, as you suggested.

From the top of my head, one solution would be:

1. Use the bullet ray casting to get the physics object, which should be faster than iterating over all visual meshes, since the physics meshes are normally simpler than the visual meshes. Also the bullet ray casting is pretty well optimized AFAIK.

2. When you get the rigid body of the object you know which game object you are dealing with. You can supply a user data pointer to the RB to access the visual mesh.

3. Do D3DXIntersect() tests on the mesh, like you do now.

This, of course, only works as far as all your objects have rigid bodies, but most will right? And those that don't can be handled separately.

This topic is closed to new replies.

Advertisement