• FEATURED

View more

View more

View more

### Image of the Day Submit

IOTD | Top Screenshots

### The latest, straight to your Inbox.

Subscribe to GameDev.net Direct to receive the latest updates and exclusive content.

# how to do ray picking in shaders

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

8 replies to this topic

### #1sobeit  Members

Posted 22 June 2013 - 04:20 AM

Hi there,

recently I was trying to implement ray cast picking using geometry shader. I want to take mouse position of screen as parameter and return the intersection point of the picking and the triangle in world space or even better in object space.

I kinda know the basic idea, but I've got some confusions.

1. I can do line-triangle intersection in geometry shader, but I want to find a way to tell geometry shader, once after intersection test succeeds, to stop doing intersection test for rest of triangles. I don't how know to do that.

2. what if there are several objects along the way of picking ray, how do I determine the nearest one. even if there is only one object, how do I determine whether the triangle face intersecting with the ray is the back or front of the object?

3. I think I have to use transform feedback to retrieve the intersecting point, do I?

I really appreciate if someone could help out on this. Thanks.

Edited by sobeit, 22 June 2013 - 04:20 AM.

### #2RobTheBloke  Members

Posted 22 June 2013 - 07:21 PM

1. Don't do that, you may miss closer intersection points on the same mesh.

2a. Sort the pick results based on depth.

2b. dot product the pick ray with the face normal. If +ve, it's a back face, if -ve it's a front face.
3. probably the best method available.

### #3sobeit  Members

Posted 23 June 2013 - 12:18 AM

1. Don't do that, you may miss closer intersection points on the same mesh.

Thank you very much. Is there any variable in shaders that it's value can persist through out one whole shading stage? like say I define a variable int counter = 0 in geometry shader, and I add a line counter++;. every time geometry shader executes, counter will increase by one.

2a. Sort the pick results based on depth.
2b. dot product the pick ray with the face normal. If +ve, it's a back face, if -ve it's a front face.

How do I record depth result in geometry shader?

Edited by sobeit, 23 June 2013 - 01:24 AM.

### #4WiredCat  Members

Posted 23 June 2013 - 01:06 AM

i think you should read depth data from depth buffer, instead of making rays

### #5sobeit  Members

Posted 23 June 2013 - 01:11 AM

i think you should read depth data from depth buffer, instead of making rays

but I need to use ray to do intersection with triangles.

### #6RobTheBloke  Members

Posted 23 June 2013 - 05:28 AM

i think you should read depth data from depth buffer, instead of making rays

Chicken and egg. Geometry has not yet been rasterised at that point in the pipeline.

1. Don't do that, you may miss closer intersection points on the same mesh.

Thank you very much. Is there any variable in shaders that it's value can persist through out one whole shading stage? like say I define a variable int counter = 0 in geometry shader, and I add a line counter++;. every time geometry shader executes, counter will increase by one.

Why would you want to do that?

2a. Sort the pick results based on depth.
2b. dot product the pick ray with the face normal. If +ve, it's a back face, if -ve it's a front face.

How do I record depth result in geometry shader?

dot product the pick ray with the intersection point, store in the output point you generate.

Edited by RobTheBloke, 23 June 2013 - 05:30 AM.

### #7sobeit  Members

Posted 23 June 2013 - 04:06 PM

sobeit, on 22 Jun 2013 - 11:18 PM, said:

RobTheBloke, on 22 Jun 2013 - 6:21 PM, said:

1. Don't do that, you may miss closer intersection points on the same mesh.
Thank you very much. Is there any variable in shaders that it's value can persist through out one whole shading stage? like say I define a variable int counter = 0 in geometry shader, and I add a line counter++;. every time geometry shader executes, counter will increase by one.

Why would you want to do that?

I want to create bool variable to store the test result of ray-triangle intersection. If intersection succeeds and dotProduct(ray, normalOfTriangle) < 0, then I can skip intersection test for rest of triangles. thanks

Edited by sobeit, 23 June 2013 - 04:11 PM.

### #8RobTheBloke  Members

Posted 24 June 2013 - 05:48 AM

I want to create bool variable to store the test result of ray-triangle intersection. If intersection succeeds and dotProduct(ray, normalOfTriangle) < 0, then I can skip intersection test for rest of triangles. thanks

What you want, and what GPU's are capable of, are two different things. GPU's do not process the triangles serially, they process them in parallel. i.e. On a GTX titan, 1 triangle test takes the same amount of time as 3000 triangle tests. If you want to turn that into a serial process (and pass the bool from one triangle to the next), you'll be making the tests 3000x slower. So yes, you could ruin the performance of your selection routine, but I'd suggest just testing the triangles in a parallel fashion. It will be quicker.

1. Run a first pass using a ray/OBB test for the mesh. If a mesh passes that test.....

2. Test *every* triangle in the mesh.

3. Sort resulting intersections by depth.

It is impossible to early-out a ray/triangle test, without generating a bunch of incorrect intersections as a result (you might stop testing when you hit a triangle 19 meters away, however if you keep testing, you might hit a triangle only 9 meters away).

### #9sobeit  Members

Posted 24 June 2013 - 01:38 PM

What you want, and what GPU's are capable of, are two different things. GPU's do not process the triangles serially, they process them in parallel. i.e. On a GTX titan, 1 triangle test takes the same amount of time as 3000 triangle tests. If you want to turn that into a serial process (and pass the bool from one triangle to the next), you'll be making the tests 3000x slower. So yes, you could ruin the performance of your selection routine, but I'd suggest just testing the triangles in a parallel fashion. It will be quicker.

1. Run a first pass using a ray/OBB test for the mesh. If a mesh passes that test.....

2. Test *every* triangle in the mesh.

3. Sort resulting intersections by depth.

It is impossible to early-out a ray/triangle test, without generating a bunch of incorrect intersections as a result (you might stop testing when you hit a triangle 19 meters away, however if you keep testing, you might hit a triangle only 9 meters away).

Thank you so much, this totally clear my confusion.

Edited by sobeit, 24 June 2013 - 01:39 PM.

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.