I want to know how to implement gluUnProject. i dont use it because gluUnProject inverts the modelview and projection matrix (i think so) but i already have them on the fly... So i only want to know the pseudo-code for my implemention of gluUnProject... Thanks

EDIT: stupid html smilies messed up my parenthesis.

Well, you may have to do some inversion of some kind.
Basically, you want to undue the transform pipeline.

Two ways - one is to concatenate all matrices (view * projection * modelview) and solve the inverse of that, to get object coords - ie Ax=b, where A is a 4x4, x is your object coords, and b is your window coords. So solve x=A^-1 * b;
Since you sound opposed to doing matrix inverses, you can undo all the transforms if you know all components of both matrices...otherwise you have to invert.

Let's say you know everything, then...

First you go from window coordinates to homogenous coordinates.

Say you have a 640x480 OpenGL window, and a viewport which fills the whole window(which is usually the case).
Then during the viewport transform, points are transformed from x(-1,1) to (0,639) and from y(-1,1) to (0,459). Z is typically unchanged (0,1).
To undo this, first you want to center your viewport at 0,0.
So translate by (-viewx/2,-viewy/2,0).
Then scale by (2/viewx,2/viewy,1). This should map all your window coordinates into homogenous space.

Now you want to undo the projection matrix.
Let's say you're using a standard perspective matrix.
This looks like this:
(row major matrix)
M[0][0]=1.0/(tan(fovy/2)*aspect);
M[0][1]=0;
M[0][2]=0;
M[0][3]=0;
M[1][0]=0;
M[1][1]=1.0/tan(fovy/2);
M[1][2]=0;
M[1][3]=0;
M[2][0]=0;
M[2][1]=0;
M[2][2]=(zFar+zNear)/(zNear-zFar);
M[2][3]=2*zNear*zFar/(zNear-zFar);
M[3][0]=0;
M[3][1]=0;
M[3][2]=-1;
M[3][3]=0;
or algebraically:
x=x/(tan(fovy/2)*aspect);
y=y/tan(fovy/2)
z=z*(zFar+zNear)/(zNear-zFar)+w*2*zNear*zFar/(zNear-zFar);
w=-z
So basically your most smallest z (znear) maps to 0, and your biggest z(zfar) maps to 1(the negative 1 turns negative zs into positive z's...because opengl looks down the negative z axis by default).
After this, you divide everything by w, so in this case(perspective) it's like dividing by -z.
kinda tricky, but essentially you use znear and zfar to reconstruct x, y, and z in world space(w should be 1 if you are doing standard transforms). Once you have x,y, and z in world space you want to undo the transforms of your modelview matrix. Again, if you don't want to compute inverses, you have to undo all your glTranslate/glRotate/glScale calls algebraically, which is not too bad...if you get stuck just compose inverse matrices(really easy, since for a transform just replace x,y,z with -x,-y,-z; for a rotate, replace it with the transpose, since the inverse of an orthogonal matrix is its transpose, and for scales, replace x,y,z with 1/x,1/y,1/z)

Hope that helps, email me (alan_gasperini@(ignore)hotmail.com) if you need any clarification.

Edited by - sjelkjd on September 11, 2001 2:43:10 AM