Quote:Original post by bzroom
You are correct in thinking you can reconstruct the world coordinates from screen position and depth (i actually clicked this thread with hopes of finding out how).

There are several methods to do it. The main two ones are either by reverse raycasting or by simple matrix math. They both give the same results (minus floating point rounding errors), and performance depends on your situation (especially in what coordsys you need your results to be).

I'll quickly outline the second method. The idea is to reverse the screenspace remapping, the perspective divide, the projection transform and the view transform. All that in the pixel shader, as fast as possible. While that seems a lot to do, it's actually very simple, because almost all operations can be concatenated in a single matrix beforehand.

First, get the current screenspace coordinates of the pixel under consideration. Note that gl_FragCoord returns values in viewport range. We first need to remap them to [0..1], so that we can use them to index the depth texture. We later need to remap them to [-1..1] range for the matrix math.

vec2 screen = (vec2(gl_FragCoord.x, gl_FragCoord.y) - viewport.xy) * viewport.zw;

Here, viewport.xy contain the lower-left offset of your viewport (usually 0,0), and zw contains the reciprocal of your screen size. Look at the values you supplied to glViewport for reference.

Next, get the depth from the depth buffer at the position of our current fragment. I assume a correctly set up and bound depth texture. This will return a depth value in the 0..1 range. Again, we will later need to remap this to [-1..1]:

float depth = texture2D(DepthTexture, screen).x;

Now comes to magic. We build a homogeneous vector from our normalized screenspace coordinates, and multiply it with the inverse transform matrix IM. Look down below on how to construct it:

vec4 world = vec4(screen, depth, 1.0) * IM;

Finally, we need to undo the perspective divide, by dividing our vector by its w component. See it as a 'dehomogenisation' ;)

world.xyz /= world.w;

world.xyz now contains the worldspace position of the current fragment.

Now, what remains is that IM matrix. Essentially, what we need is a matrix that first remaps our [0..1] coordinates to [-1..1], then undos the projection, and finally undos the view transform (camera matrix).

Assuming column major order, the matrix is constructed as follows:

IM = inverse(ProjectionMatrix * CameraMatrix) * RemapMatrix;

RemapMatrix is a simple scale + translate matrix, that will remap an input vector from [0..1] to [-1..1] range.

Quote:Original post by Yann L

Quote:Original post by bzroom
You are correct in thinking you can reconstruct the world coordinates from screen position and depth (i actually clicked this thread with hopes of finding out how).

There are several methods to do it. The main two ones are either by reverse raycasting or by simple matrix math.

I'll cover the first one, which is extremely easy.

1) find the 4 view-space corners of the camera frustum (the 4 corners of the visible far plane) - this is application specific, but if you're using a render library (like Ogre [grin]), changes are you'll have something like Camera::getFrustumCorners() [wink]

2) pass in these 4 corners as normals/texcoords/tangents/etc. to the corresponding 4 vertices of your screen-space quad

3) interpolate this value from the quad vertex shader down to the pixel shader using a texcoord

4) normalize it there, and multiply by your depth to find the view space position. if you want the world-space position, multiply this value by the inverse of your view matrix

(depth should be the distance from the fragment to the camera, that is, the length of the view-space fragment position - you do not want post-projection depth in this case)

There's another way to do it, by using the .z coordinate of the fragment instead of its length, MJP will probably cover that [grin]. But both versions result in practically the same math.