recently I was trying to implement ray cast picking using geometry shader. I want to take mouse position of screen as parameter and return the intersection point of the picking and the triangle in world space or even better in object space.
I kinda know the basic idea, but I've got some confusions.
1. I can do line-triangle intersection in geometry shader, but I want to find a way to tell geometry shader, once after intersection test succeeds, to stop doing intersection test for rest of triangles. I don't how know to do that.
2. what if there are several objects along the way of picking ray, how do I determine the nearest one. even if there is only one object, how do I determine whether the triangle face intersecting with the ray is the back or front of the object?
3. I think I have to use transform feedback to retrieve the intersecting point, do I?
I really appreciate if someone could help out on this. Thanks.
Edited by sobeit, 22 June 2013 - 04:20 AM.