Hello everyone

On my tablet i render my scene to a depth buffer and read a pixel from it to determine my current position.

- I visualized the depth buffer -> all is fine, vertex shader:
[source lang="cpp"]varying float camDistance;

void main() {
vec4 tmpPos = vec4(vPosition0.xyz, 1.0);
vec4 posCamSpace = matProj * matView * tmpPos;
camDistance = (posCamSpace.z - 0.1) / (800.0 - 0.1);

gl_Position = posCamSpace;
}[/source]

i hardcoded the values just to be sure!

- I visualized the depth read from the buffer -> It is correct for the nearest points and for the points far away, thats the code i use:
[source lang="java"] ibuff.position(0);
GLES20.glReadPixels((int)touchPos.x, (int)touchPos.y, 1, 1, GLES20.GL_RGBA, GLES20.GL_UNSIGNED_BYTE, ibuff);
ibuff.position(0);
int color = ibuff.getInt() & 0x000000FF;
float depth = color / 256.0f;[/source]

- When i check my coordinates i see that it uses nearly the position of the camera for all 3 components and when i move my finger shifts the coorect coordinates but like 0.001 per pixel like so it seems that it unprojects the point no matter what depth i enter to the near plane.

- If I use depth 1 every time unprojection is done nicely and works like a charm if i move my finger, all coordinates are correct and change correct.

I used gluUnProject to unproject my values as well as my own implementation. With gluUnProject:
[source lang="java"] GLU.gluUnProject(touchPos.x, touchPos.y, depth,
getActiveCamera().getMatView().getFloats(), 0,
PerspectiveMatrix.getFloats(), 0, viewport, 0, coords, 0);[/source]

touchPos.y of course is transformed with the viewport (touchPos.y = viewport[3] - touchPos.y).

Thats my own implementation:
[source lang="java"] public static Vector3 unproject(Vector3 v, Matrix invView, Matrix invProj, int[] viewport) {
Matrix matCombo = Matrix.multiply(invView, invProj);

float vx = viewport[0];
float vy = viewport[1];
float vw = viewport[2];
float vh = viewport[3];

float[] rhsVec = {
((2 * (v.x - vx)) / vw) - 1,
((2 * (v.y - vy)) / vh) - 1,
2 * v.z - 1,
1
};

float[] result = new float[4];
android.opengl.Matrix.multiplyMV(result, 0, matCombo.getFloats(), 0, rhsVec, 0);
float d = 1 / result[3];
return new Vector3(result[0] * d, result[1] * d, result[2] * d);
}[/source]

What did i forget? I used gluUnProject on my Desktop with glReadPixels (but GL_DEPTH_COMPONENT which is not available on OpenGL ES) and it worked there!

Thank you for any help
Plerion

The glu version is giving incorrect results also? Can you provide some numbers to give an idea of how incorrect it is?

Sure, i made some screenshots from my tablet. Im sorry the font seems to be screwed up a bit, i hope its still readable, as i directly wrote into the screenshots! The big arrow always points to the location where my finger pressed. All the results come from the version with gluUnProject.

Are these calls to unproject taking the model (world) matrix of that object into account? The original gluunproject call takes the modelView and projection matrix. I cant tell if the view matrix you pass includes the model matrix of the object being rendered.

Hey

The calls to the drawing for depth and visual are like that:
[source lang="java"] prog.setMatrix("matProj", Game.Instance.getGraphicsDevice().getProjMatrix());
prog.setMatrix("matView", Game.Instance.getGraphicsDevice().getActiveCamera().getMatView());[/source]

where getProjMatrix() returns the perspective matrix (PerspectiveMatrix) i pass to gluUnProject, getActiveCamera().getMatView() is what i use for gluUnProject too. So i use the absolut same matrices (there is no model matrix, they already have their absolute coordinates in the world).

This problem is starting to driving me mad... I now even put those goddamn matrices hardcoded for the shader and the gluUnProject, still gluUnProjects gives me complete garbage as result. The view-matrix seems to be correctly interpreted but the perspective matrix seems to be treated wrong. After closer inspection i realized that also for depth = 1 the results are rubbish. My cameras left to right axis is parallel to the y-axis, when i go with a constant depth of 1 from left to right it gives me a difference of 200 units, after calculating a bit that leads to a horizontal field of view of 15° which is clearly not the case. I do it the following way now:
[source lang="java"] GLU.gluUnProject(touchPos.x, y, depth,
Matrix.LookAt(getActiveCamera().getPosition(), getActiveCamera().getFront(), getActiveCamera().getUp()).getFloats(), 0,
Matrix.Perspective(45.0f, mWidth, mHeight, 1, 800.0f).getFloats(), 0, viewport, 0, coords, 0);[/source]

With the following functions:
[source lang="java"] public static Matrix Perspective(float fov, int width, int height, float zNear, float zFar) {
Matrix ret = new Matrix();

android.opengl.Matrix.perspectiveM(ret.mMatrixFloats, 0, fov, ((float)width / height), zNear, zFar);

return ret;
}

public static Matrix LookAt(Vector3 eye, Vector3 target, Vector3 up) {
Matrix ret = new Matrix();

android.opengl.Matrix.setLookAtM(ret.mMatrixFloats, 0, eye.x, eye.y, eye.z, target.x, target.y, target.z, up.x, up.y, up.z);

return ret;
}[/source]

To conclude this topic:
I havent been able to actually solve the problem but i circumvented it. I do now all calculations by hand and the results are as i expected: Perfectly right!
[source lang="java"]public class Unproject {

public void setPerspective(float fovy, float width, float height, float zNear, float zFar) {
this.zNear = zNear;
this.zFar = zFar;
this.fovy = fovy;

fovh = (width / height) * fovy;
}

public void setLookAt(Vector3 eye, Vector3 forward, Vector3 up, Vector3 right) {
camEye = eye;
camForward = forward;
camUp = up;
camRight = right;

camRight.normalize();
camUp.normalize();
camForward.normalize();
}

public void setViewport(int[] viewport) {
this.viewport = viewport;
}

public Vector3 process(Vector3 screen) {
float vx = viewport[0];
float vy = viewport[1];
float vw = viewport[2];
float vh = viewport[3];

screen.x = ((2 * (screen.x - vx)) / vw) - 1;
screen.y = ((2 * (screen.y - vy)) / vh) - 1;

float z = screen.z;
float depth = zNear + (zFar - zNear) * Math.abs(z);
float factor = 1;
if(z < 0) {
factor = -1;
depth *= -1;
}
float lenh = (float)Math.tan(fovh * Math.PI / 360.0f) * depth;
float lenv = (float)Math.tan(fovy * Math.PI / 360.0f) * depth;

Vector3 target = Vector3.add(camEye, Vector3.mul(camForward, depth * factor));
target = Vector3.add(target, Vector3.mul(camRight, screen.x * lenh));
target = Vector3.add(target, Vector3.mul(camUp, lenv * screen.y));

return target;
}

private float zNear, zFar;
private float fovy, fovh;
private Vector3 camEye, camForward, camUp, camRight;
private int[] viewport;
}[/source]