# 3D mouse coords: gluUnproject again

## Recommended Posts

Hi, I've got a OpenGL scene. A player/camera, which position is (64,TerrainHeight,64) and a Terrain starting at (0,0,0) which has a size of 128x128. Now I want to get the mouse coords in 3d space. I use following code:
Vector3 Engine::Get3DMousePosition()
{

POINT mousepos;
GetCursorPos(&mousepos);
double dx; double dy; double dz;

GLdouble modelview[16], projection[16];
GLint viewport[4];
float z;

glGetDoublev( GL_MODELVIEW_MATRIX, modelview );
glGetDoublev( GL_PROJECTION_MATRIX, projection );
glGetIntegerv( GL_VIEWPORT, viewport );

glReadPixels( mousepos.x, viewport[3] - mousepos.y, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &z);
//gluUnProject(mousepos.x, iRealY, depth[0], model_view, projection, viewport, &dx, &dy, &dz);
gluUnProject( mousepos.x, viewport[3] - mousepos.y, z, modelview, projection, viewport, &dx, &dy, &dz);

return Vector3(float(dx), float(dy), float(dz));
}


All I get are weird results which range e.g. for the x coordinate from < -1000 to > 900. I'm confused and don't know what I am doing wrong. Also some internet sources say you get a Ray others say you get the true 3d mouse position... :( Oh, I found out that it must have something to do with the fact that I set the Mouse Cursor to a fixed point for Camera rotation (may the rotation could mess it up too?) [Edited by - starvinmarvin on December 18, 2008 4:28:36 PM]

##### Share on other sites
resle    129
Looking at my old GL code for that: try getting the matrices as FLOAT instead of DOUBLE. Might work. Might now. I migrated to DX, I don't remember :)

##### Share on other sites
steven katic    275
I would look at what GetCursorPos() provides to you exactly.
e.g. seems like it may be providing the screen co-ordinates.
where as you probably want the co-ordinates relative to the 'window' that is your viewing plane.

?

##### Share on other sites
swiftcoder    18437
Quote:
 Original post by starvinmarvinAlso some internet sources say you get a Ray others say you get the true 3d mouse position... :(
If you read back the actual z-buffer value under the mouse, and pass it to gluUnProject (as you are currently), then you get a 3D position. If instead you pass a fixed z-value, you will get some sort of ray.

##### Share on other sites
resle    129
Quote:
 Original post by steven katicI would look at what GetCursorPos() provides to you exactly.

Right: correct me if I am wrong but GetCursorPos() returns coords relative to the whole screen. In case your application is windowed you should subtract the window top-left coords, something like..

pos = GetCursorPos()
pos.x = pos.x - window.x
pos.y = pos.y - window.y

##### Share on other sites
Thanks. I got it working so far, I was calling the function before camera rotation and translation. But I still don't get the correct results. And yes, I'm in windowed mode. Tried viewport[3] - mousecoord.y too. I get slightly better results when doing something like: mousecoord.x * screenwidth and mousecoord.y * screenheight,