# How to convert screen coordinates to OpenGL coordinates

This topic is 3519 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

Hi! I'm trying to develop a small OpenGL application on Adroid, using OpenGL ES 1.0 or version 1.1. I would like to know how to convert the screens touch coordinates to OpenGL coordinates, so I can move an object while my finger is moving on the screen. I have looked at GluUnProject but unfortunately it seems to not work as intended. Someone told me that it's not supported in this version of Android. Is there a good way how to solve this problem without using GluUnProject? I have looked at this article but it uses GluUnProject as a solution. http://nehe.gamedev.net/data/articles/article.asp?article=13 Thanks

##### Share on other sites
When you say that gluUnproject doesn't work, do you mean that you get nonsensical results, results that "sort of might be right but aren't" or that it crashes/doesn't link?

Because gluUnproject might be working fine, but being fed the wrong input; you need to do it twice really, once for the front of the view volume, once for the back and then to look along that line looking for an object. If you've got some results, but they're just odd, then you might have the inputs squiffy.

Your other option is "picking". Basically you select a point, render your scene tagging each object and when you paint that point, the system remembers which object did it and you can query that later. This is a bit on the expensive side -- although you can simplify it by rendering simpler objects, turning off lighting/texturing all that jazz. Only rendering the objects that could be touch targets, doing some culling before rendering, etc.

##### Share on other sites
It really helps to understand the math behind OpenGL coordianate transformations. Then you can apply these in reverse to get the result.
NB! as screen coordinates are 2D, you cannot get single 3D coordinate by applying reverse transform, but instead a line. Also, object, world, eye and clip coordinates are 4-dimensional.
Usually the following are involved:

1. World coordinate system (normally tied to level, with z pointing upwards, x and y horizontal
2. Object coordinate system. Object is modelled in private coordinate system (usually 0,0,0 being center or under feet). Object placement in world coordinates is specified as object transformation matrix M.
2. Eye coordinate system. This is local coordinate system attached to camera, camera being at 0,0,0 facind towards negative z and x being horizontal, y vertical. Camera placement in world coordinate system is described by view transformation matrix V.
3. Projection matrix Proj. It transforms coordinates from eye system to new coordinate system called clip coordinate system.
4. Perspective division. In this step clip coordinates x,y,z are divided by clip w to get perspective, resulting in device coordinates.
5. Viewport. This is defined by 6 numbers - x0, x1, y0, y1, z0, z1 - the boundaries of visible are in screen (including visible depth range). Device coordinates are simply scaled/translated so -1,-1,-1 - 1,1,1 cube fills the whole viewport

You need some library for matrix math (multiplication, inversion) and also to know the values of any involved matrix (either store these themselves or read back from OpenGL).
Now the math:Po, Pg, Pe, Pc - point coordinates in object, world, eye, clip coordinate system                 (4-dimensional, Wo,Wg,We = 1)Pd, Pw - point coordinates in device, window coordinate system (3-dimensional)M - object transformation matrixV - eye (camera) transformation matrixProj - projection matrixXYZ0, XYZ1 - screen boundaries (usually X0 = 0, Y0 = 0, X1 = width, Y1 = height, Z0 = 0, Z1 = 1We assume all matrixes to be in OpenGL formatPg = M * PoPe = V'* PgPc = Proj* PePd = Pc/WcPw = [(XYZ1 - XYZ0) / 2] * Pd + (XYZ1 + XYZ0) / 2;In opengl Object and View matrixes are usually combined into single modelview matrixMV = V' * Mand thus:Pe = MV * PoNow lets start the reverse process:Xw, Yw - your screen coordinatesPick one arbitrary Z - for example Zw = 0 (meaning that point is on near plane)Xd = [Xw - (x1 + x0) / 2] / [(x1 - x0) / 2]Yd = [Yw - (y1 + y0) / 2] / [(y1 - y0) / 2]Zd = [0  - (z1 + z0) / 2] / [(z1 - z0) / 2]Now we have to know Wc to proceedWe know that We = 1, andPd = Pc/Wc[Xc,Yc,Zc,Wc] = Proj * [Xe,Ye,Ze,1][Xe,Ye,Ze, 1] = Proj'* [Xc,Yc,Zc,Wc]Solving this we get:Wc = 1 / (Proj'[3] * Xd + Proj'[7] * Yd + Proj'[11] * Zd + Proj'[15]);Xc = Xd * WcYc = Yd * WcZc = Zd * WcPe = Proj' * PcPg = V * PePo = M' * PgorPo = MV' * Pe

So you get single point on near plane. To get a line, take this and also either:
- camera center point (for perspective camera)
- point some distance along camera local Z coordinate (for isometric camera)
and construct line from those 2 points. This is the world line, that represents a point on your screen. Use this line to do picking, moving etc.

##### Share on other sites
Thanks a lot for the explanation.

this line: Pe = V'* Pg

why V' ?

Thanks

• 17
• 9
• 15
• 60
• 84