# OpenGL Converting screen coordinates to 3D world coordinates

This topic is 2022 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

I'm writing a game which could loosely be described as a strategy game, with a typical camera - it looks down onto the world from a high angle with a perspective projection.  The player will interact with the game by tapping or clicking things on the "ground" and will also be able to move the camera a bit.  For various reasons I'm writing my own engine instead of using a ready-made one.  It's based on OpenGL ES 2.0 (or an equivalent subset of OpenGL for PCs) with glm for the maths.

With the help of a diagram and schoolboy trigonometry I managed to come up with an equation for calculating the minimum zfar value to use in glm::perspective to render the ground at its farthest from the camera (at the top of the screen) but I don't know how to work out where a point on the screen corresponds to on the ground.  I think glm::unproject will be useful but the trouble is I don't have a meaningful z-coordinate to plug in to glm::unproject.  I know you can read the depth buffer to get this, but I also want to be able to work out which bits of the ground are visible (at the four corners of the viewport) and use this to limit the camera's movement (and to work out which bits of the terrain are outside the view and don't need to be rendered), so it would be better if I could do this before rendering anything.

I thought I could get a line joining the same NDC X and Y on the near and far clipping planes then use my diagram/equation to work out where this line crosses my ground plane, but it didn't work.  I think the main problem is that I assumed a linear mapping of Z between world space and device space, and I don't think this is the case for a perspective projection.  I'm also not sure of the Z values for the near and far planes to use as input to glm::unproject. Is it 1.0 for far and -1.0 for near?

Rambling on a bit, I had a lot of trouble understanding the perspective divide.  Am I right in thinking this is an extra step OpenGL automatically performs after the matrix transformations, and it just converts [x, y, z, w] into [x/w, y/w, z/w]?  And that an orthogonal projection matrix sets w to 1 and a perspective one sets it to z?  But z from which "space"?

##### Share on other sites
I think you can multiply the coordinates with the inverse viewprojection matrix

##### Share on other sites

I'm not familiar with what functions GLM has available. But the principle for picking the ground mesh should go something like:

Assuming that the depth buffer is set -1 to 1 for near to far planes (see link below):

Unproject a 3D vector posNear(mousex, mousey, -1.0) to get the world position at the near plane.

Unproject a 3D vector posFar(mousex, mousey, 1.0) to get the world position at the far plane.

3D vector rayDir = normalize(posFar - posNear)

You should be able to pick the ground mesh with posNear and rayDir.

If the ground is not a mesh, but a plane: solve for d where posNear + d*rayDir is a point in the plane.

EDIT: This post says depth buffer values are -1 to 1.

Edited by Buckeye

##### Share on other sites

If you are using the results of this unprojection for doing anything else than pure visuals, it might make more sense to raycast into your scene on the CPU. There are probably situations when you want your meshes outlines to not correspond with the actual borders of the object.

##### Share on other sites

@cozzie: I think multiplying by the inverse of the MVP is basically what glm::unproject does. It's based on a similar function in GLU.

If the camera was looking directly down the vertical it would be relatively easy because the ground would have a fixed Z in device space.  But my camera is tilted (as if on the nose of a diving aeroplane) so the ground's Z in device space varies with Y.

Buckeye and Waterlimon, you confirm what I was thinking about getting a line between the same (X, Y) coordinates at Znear and Zfar, but I don't know how to work out where this ray intersects my terrain or other objects in world space.  Or even what the values are for Znear and Zfar are in the input to glm::unproject. I think it should be NDC, so -1 and 1 respectively?

##### Share on other sites

Sorry, accidental double post.

Edited by realh

##### Share on other sites

Disclaimer: I'm not familiar with GLM, in particular.

However, once you've calculated posNear and rayDir (see my post above), you may be able to perform intersection tests with spheres and triangles using something as described on this page (which I googled.)

For quick culling, you may want to create bounding spheres for your objects. I would think an "intersect sphere" function would be very quick. If you get a hit, and you need to fine-tune the hit location, look for intersections of the line with triangles in the object.

With regard to the near and far parameters for setting up the pick vectors: did you look at the link I provided above? What else did you find when you googled?

You should be able to do a quick test by using -1 as the z-component of the posNear vector, unproject it and check that it's the same as your eyepoint or camera position.

##### Share on other sites

With regard to the near and far parameters for setting up the pick vectors: did you look at the link I provided above? What else did you find when you googled?

The first link raises an important point about the depth buffer using the range [0, 1] whereas NDC is [-1, 1], so thanks for posting it. The main thing I found by Googling was that you can read back the depth buffer to get the Z coord at a certain screen X, Y, but I couldn't find much about doing my own maths. I've realised the non-linear relatinship between Z in NDC and world space must be a red herring because my ray was in the latter, so a simple linear interpolation should find where it crosses Z=0 (assuming that's where my ground is). My mistake was to use 0 instead of -1 for Znear in NDC.

You should be able to do a quick test by using -1 as the z-component of the posNear vector, unproject it and check that it's the same as your eyepoint or camera position.

Isn't -1 on the near clipping plane rather than at the camera?

##### Share on other sites

Isn't -1 on the near clipping plane rather than at the camera?

What did you get when you tried it? Something other than the camera position? To answer your question: yes, but it should be very close if you use the camera position to set up the view.

Edited by Buckeye

##### Share on other sites

I haven't tried it yet. My Znear is quite a long way from the camera because in this game the field of view is wide in relation to the maximum height of visible objects (although it's a perspective projection this would probably be classed as 2.5D) and I read it's best to make the near-far range as small as possible to make the most of limited depth buffer precision.

• ### Game Developer Survey

We are looking for qualified game developers to participate in a 10-minute online survey. Qualified participants will be offered a \$15 incentive for your time and insights. Click here to start!

• 13
• 18
• 15
• 10
• 9