Jump to content
  • Advertisement
Sign in to follow this  

OpenGL Manual Ray Picking

This topic is 2806 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts


I have encountered a rather interesting problem in a game I am developing that uses OpenGL. The problem is that the 3D scene is rendered to a multisampled FBO, and because of this the traditional "glReadPixels/gluUnproject" method of getting a picking-ray does not work (because glReadPixels() is needed to read the depth for gluUnproject, but you can't read the depth from multisampled FBO). So, I am looking into alternative methods to generate picking-rays.

If I where developing a FPS game the solution would be simple: Just use the view vector of the camera. But because I am developing an RTS game, I need to be able to calculate picking-rays from anywhere the mouse may be at on the screen.

With that said, is there a quick 'n easy way to manually calculate a picking ray using the OpenGL modelview and projection matrices and the mouse coordinates? Perhaps there is a way with the zNear/zFar and the view vector of the camera and the mouse coordinates? etc etc? Because of implementation issues, I would like to manually calculate the picking-rays.

Thanks beforehand!

Share this post

Link to post
Share on other sites
1. Compute the normalized, symmetric position [ v[sub]x[/sub] v[sub]y[/sub] ] on the view from the pixel co-ordinates [ p[sub]x[/sub] p[sub]y[/sub] ] and pixel extent [ N[sub]x[/sub] N[sub]y[/sub] ] like so:
v[sub]x[/sub] := ( p[sub]x[/sub] + 0.5 ) / N[sub]x[/sub] - 0.5
v[sub]y[/sub] := ( p[sub]y[/sub] + 0.5 ) / N[sub]y[/sub] - 0.5
This formula let the ray pass through the center of the pixel.

2. Unnormalize that position using the (normalized) up vector c[sub]y[/sub] and right vector c[sub]x[/sub] from the camera, and the extent [ S[sub]x[/sub] S[sub]y[/sub] ] of the view in the world.
p[sub]1[/sub] := S[sub]x[/sub] * v[sub]x[/sub] * c[sub]x[/sub] + S[sub]y[/sub] * v[sub]y[/sub] * c[sub]y[/sub]

3. Determine the ray from the camera position p[sub]c [/sub]and forward vector c[sub]z[/sub]
a. for an orthogonal projection as
R(t) := p[sub]c[/sub] + p[sub]1[/sub] + t * c[sub]z[/sub]
[size=2]b. for a perspective projection with view distance D as
[size=2]d[size=2] := D * [size=2]c[size=2][sub]z[/sub][size=2] + [size=2]p[size=2][sub]1[/sub][size=2]
[size=2]R[size=2](t) := [size=2]p[size=2][sub]c[/sub][size=2] + t * [size=2]d[size=2] / |[size=2]d[size=2]|

You may need to use the correspondence of the view distance D, (horizontal) view size S[sub]x[/sub], and (horizontal) opening angle a
[size=2]tan( a ) = 2D / S[sub]x[/sub]
[size=2]to determine D.

[size=2]Check for ray-volume hits using the above ray as usual ... well, after checking the above math twice ;)

Share this post

Link to post
Share on other sites
This is a bit of a crutch... but why not glBlitFramebuffer into a different (1x1 pixel) FBO and read back from that?

This will let you reuse your existing code, and the blit should not be *that* much of an overhead. You'll likely want to readback using a PBO anyway to hide the latency, and so a little bit more doesn't really matter (especially since the latest generation nVidia consumer cards throttle readbacks down artificially, so proper use of async transfers is even more important).

Share this post

Link to post
Share on other sites
Don't do the picking in model space. Do the picking in screen space and have the GPU do the heavy lifting.

There are basically two questions involved in picking. Which is the object which best fits "being at X,Y on the screen". The second is "which objects lie in the rectangle X1,Y1 X2,Y2".

You almost certainly have a limited set of objects to consider (soldiers yes, trees no)..

So.. write a couple of shaders which answer those questions and write into feedback buffers. They'll have all the access needed to the projection matricies and so on. You just need an array of verticies to pass in which contain world(XYZ) coords for the objects and size indicators (again in world). (You may well have a suitable vertex array lying about anyway, because it may conveniently be part of your instancing system). The shaders can happily burble away doing projections and square roots and matrix maths... and it'll all happen super fast somewhere else.

The output for the first case is a single number which represents (for the first question) a Z depth for the object or a value which means no contention. When complete, sort this array, first element is your closest object which contains screen(XY).

For the second case the output can just be a 0 or 1 "yes/no" indicator as to whether the projected centre is contained in screen(X1,Y1,X2,Y2). You just scan that array.

Job done.

If you're really cunning about it, you can arrange for the main CPU to be doing something else while the GPUs do that work.

Share this post

Link to post
Share on other sites

This is a bit of a crutch... but why not glBlitFramebuffer into a different (1x1 pixel) FBO and read back from that?

@samoth: An excellent idea, but I can't because my app is multithreaded, and OGL gets grumpy if you make calls to the OGL api from different thread. The 1st thread handles the OGL rendering, while the 2nd handles the game logic. Its the second that needs the picking rays, but it can't make calls to the OGL api. So, to avoid passing picking requests/results between threads I would rather just manually calculate them :).

@haegarr: Awesome! That's just what I was looking for!

Although I am still going to implement haegarr's method of calculating a picking ray, I managed to find an alternative method. It turns out that you don't need the depth to calculate a picking ray, so a call to glReadPixels is unnecessary. You just need the modelview/projection matrices and the mouse coords. From there you can call gluUnProject twice: once with a depth of 0.0 for the origin of the ray, and again with the depth of 1.0 for getting the end of the ray. Now you have a classic start/end line that can be tested against bbox's and such. If you need a origin and direction vector rather than a line, you can use start as the origin and normalize(end - start) for the direction vector. This leaves one problem: still can't call gluUnProject() from the second thread.... The fix for this is a custom implementation of gluUnProject() based off of the one here: http://www.opengl.or...UnProject_code.

For those of you who may encounter the same and/or similar issues as this, here is my solution to the problem: http://pastebin.com/9VLEgtpU

Thank you haegarr, samoth, and Katie for your help!

Share this post

Link to post
Share on other sites
Sign in to follow this  

  • Advertisement

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!