I am trying to devise an algorithm to which object in my 3D world the user clicked. If two targets are close to eachother, the one closer to the near plane will be chosen. Here is my thought process for how to build this algorithm, and I just want to make sure I'm on the right track before I get too deep into this
My near distance on my view frustum is 1.0
1) Determine the direction vector between the origin and the spot clicked on the screen. This is done by calculating the following values:
[source lang="cpp"]float rayX = (position.x / halfWidth) - 1;float rayY = -(position.y / halfHeight) - 1;float rayZ = 1.0f;[/source]
This produces a ray with values -1 <= x <= 1 and -1 <= y <= 1. The z-coordinate is equal to the distance to the view frustum near plane.
2) Transform the near plane by the inverse transpose of the (modelview * projection) matrix
3) Calculate the intersection point between this ray and the transformed near plane.
4) From this intersection point create a ray with the intersection point as the start point of the ray, and the vector (0, 0, 1) as the direction vector.
5) For each object that is clickable, test if the ray intersects the bounding volume of the mesh
5a) if more than 1 match is found, the intersection point with the closest z coordinate to the near plane is chosen.
Am I totally off here? Am I close but need some tweaking? Am I greatly overcomplicating this problem? Any help would be great.