I've been searching high and low on google for hours on how to perform picking without the use of OpenGL's selection buffer. I can do this easy with Direct3D (as long as I use D3DX unfortunately), but I can't find anything for OpenGL! I need it for my upcoming 3rd person shooter game.

I already understand how the process works after reading up on it at toymaker.com (which is Direct3D only), and I can do most of the work myself easily by using the algorithm from the old Dx9sdk picking sample, but what is the equivelant of D3DXMatrixInverse for OpenGL? And can I use the modelview matrix instead of the view matrix itself for OpenGL's sake? I searched for that two and didn't find anything. This sucks. Any ideas? Thanks.

blueshogun96,

If you can read Python, feel free to download the following program which I believe contains the requisite algorithm:

http://home.hiwaay.net/~gedesch/samples/tutorial3c.py

Specifically, the getSelectVolume(n,mx,my) function presents the conversion of the pick box select volume from viewport coordinates to world space coordinates. The push() procedure contains the algorithms to compute the select volume center-line vector and conversion of the observer's eye position from viewport to world space coordinates.

Respectfully,
G. E. Deschaines

Since I'm a bit rusty, it might take a bit of work to form a view matrix out of my 3rd person camera code. I found out that for finding a tutorial on writing a 3rd person camera was also painfully hard, but I did find this one here: http://www.swiftless.com/tutorials/opengl/camera3.html. It's not a bad piece of code (even though the mouse movement isn't very smooth), but it was done the easy way using glRotate and glTranslate.

There's another one at dhpoware, but that one was a bit too advanced for me to integrate into my code (last time I tried). I probably should try again though.

I appreciate the links, but it will take me a moment to digest all of this and integrate it into my code.