I have a terrible problem. I have a 3D terrain made up of a grid of vertices and attached triangles using D3D. The terrain is also attached to a height-map to give it some natural height and variability.
Now, using the mouse and keys, you can fly around the map and get any vantage point you wish. Although, your mouse also acts like the usual cursor.
What I need is : When the mouse is moved around the screen, I want to know exactly where on the map it is pointing. Not just the vertex or the specific triangle, but where exactly on the triangle.
This is a horrible problem I think, anybody have any suggestions on how to do this in D3D ?
Wow, I just grappled with this problem a couple weeks ago. Anyway I wrote some code to do it and I could just paste it here but there's a bit too much and it does use some strange stuff. Goto http://members.xoom.com/mutex0/defyengine and download the sources.
The mouse-triangle selection code is at the locations listed below.
The code backtransforms the mouse screen coordinates into a ray (GetMouseRay). It then loops through either all of the faces defined or all of the points (GetPointAtCoord and GetFaceAtCoord) and calls either CDefyPoint::IntersectPoint or CDefyFace::IntersectPoint, which does the actual intersection calculation.
The face intersection code is a bit easier to understand. First it gets 3 points from the point list for the face. Then it converts the points to a Triangle3 structure (TriangleFromVectors in DefyPoint.cpp).
After that it calls LineTriangleIntersection (linetri.cpp) which calculates the intersection point of a ray and a triangle. Linetri.cpp was obtained from Magic Software at http://www.cs.unc.edu/~eberly.
There was a post on one of the forums here about this and most of the info was from a post by someone (can't remember name).
Hey Spider, About a week ago, I offered a routine for finding whether a 2d pixel is inside a 2d triangle. You just have to pass the pixel to the function, and the three points of the triangle. If you want the code, email me at email@example.com.
If you can get to the direct3d transformations after they''ve been transformed, you can use this. If you don''t know how to, direct3d has a function called PICK (look it up in the SDK) which claims to tell you if a point is within a triangle on an execute buffer. I''ve never used PICK myself, so I don''t know how it works, or even if.
BTW, if you know how to get at d3d vertices after the transform, drop me some code! I know processvertices does it, but I can''t seem to make it work (I use old-fashioned execute buffers, not drawprimitives).