Quote:Original post by spek >> the reason why its not always desirable to store position is because it can require more bandwidth and memory space to store and retrieve the data Sounds logical. But as far as I know, you can only use a 16/32F format with 4 channels anyway (in OpenGL, not sure about Direct3D). Storing only the depth could be very usefull for deferred shading since there is a lot of other data to store as well. But in the examples I saw, the RGBA texture was only used for storing the depth. So I though maybe I was doing something wrong, maybe the pixel position is not converted to "world space", but another space... Or something.

I have no idea which formats are supported in GL, in D3D I use a very convenient singlechannel 32bit floatingpoint format for storing my depth. As for storing depth as RGBA8, I remember that fabio policarpo's
deferred shading tutorial does this and uses a pair of functions for encoding and decoding a single floatingpoint value in RGBA8 format. I'd imagine this should work in a pinch, at the cost of some shader math.
Quote:Original post by spek As for the depth z/w part, let's see if I'm doing it exactly right:
// vertex shader out.pos = mul( modelViewProjection, in.vertexPos ); // Pixel shader out.depth = out.pos.z / out.pos.w;
If I look at the result of this (stored as a texture), it seems to be ok. Although I can't see if the depth is 100% correct of course.

That is right for storing depth as z/w. It should look "right", as this is precisely what gets stored in the zbuffer. But in this case its not really so much a case of "right", and more a case of "different". The difference between linearized eyespace depth (which you need to use for reconstruction position from the frustum corners) and z/w is that z/w is scaled between the far plane and the near plane of your projection frustum. So for example if your projection frustum had a minimum depth of 1.0 and a max depth of 1000.0, a z/w value of 0.0 would correspond to a viewspace depth of 1.0 and a z/w value of 1.0 would correspond to a viewspace depth 1000.0f. 0.5 would correspond to 500.5, and so on. Now for what I use, for calculating position from the frustum coordinates, the depth is normalized to a range between 0.0 and the depth of the far frustum plane. So in our example from before, a depth value of 0.0 would correspond to a viewspace depth of 0.0, while 1.0 > 1000.0 and 0.5>500.0. The difference is subtle, but important. My code for calculating this depth value looks something like this:
//vertex shader
OUT.viewSpacePos = mul(IN.position, worldViewMatrix);
//pixel shader
OUT.depth = viewSpacePos.z / cameraFarZ; //cameraFarZ is the the z value of the far clip plane of the projection frustum
Quote:Original post by spek
As for the second part, at least my "3dPos = cameraPos + viewVec * depth" was correct :). >> Is in.vertexPos.xy the position in object space or in world space? Normally I would multiply the in.vertexPos with the ModelViewProjection indeed. But in this case, it isn't necesary. I render a quad with corner coordinates (1,1 .. +1,+1), since it needs to stay in front of the camera ('HUD quad'). The vertex shader just passes these coordinates. I don't know in which space they are then, "Projection Space"?. But...... maybe this won't work for calculating the viewVector.
And how to calculate the farplane corners? Sorry for these dumb questions, but all that matrix and space stuff is really difficult to me. Talking about it, what exactly is clipspace, screenspace and viewspace? I understand world and object space, but these others are confusing. You get that when a point is transformed into the view frustum or something (where 0,0,0 would be the camera position?) ?
Thanks for helping MJP, Rick 
Let's start with the different coordinate spaces. I'll try to explain as best as I understand, please forgive me if it turns out my own understanding is inaccurate:
Viewspace (also known as eyespace) is a coordinate system based on the location and orientation of the camera. The camera position is always <0,0,0> in viewspace, since its centered around the camera. This also means that if a certain point is 5 units directly in front of where the camera is facing, it will have a viewspace position of <0,0,5>. Since viewspace is just a translation and a rotation of your original worldspace, viewspace can be used for performing lighting calculations. Other spaces that utilize the perspective projection matrix can't be used for this, since perspective projection is a nonlinear operation.
Clipspace is the result of transforming a viewspace position by a perspective projection matrix. The result of this is not immediately usable, since the x y and z components must be divided by the w component to determine the point's screen position. This screen position is referred to as normalized device coordinates. In the vertex shader, the clipspace position is output since it can still be linearly interpolated in this form. Once you perform perspective division (divide by w), you can no longer interpolate which is needed for rasterization.
So getting back to your code...Your fullscreen quad's vertex coordinates are going to be in some form of postperspective format (I assume they're pretransformed), and this means they're not directly usable calculating vectors since they're not in worldspace or viewspace.
As for your frustum coordinates, they're very easy to calculate. You should check out
this article which explains how the frustum works better than I could, after that you should understand how to calculate any corner of the frustum. The corner I use in my implementation is referred to as "ftr" in that article.
EDIT: this is how I actually perform viewspace reconstruction in my renderer:
In depth pass:
multiply objectspace position by worldView matrix to get viewspace position
divide viewspace z by the farZ of the view frustum
In lighting pass:
pass in coordinate of upperright vertex of the view frustum's far clip plane
viewDirection.xy = (projectedPos.xy/projectedPos.w) * frustumCoord.xy
viewDirection.z = frustumCoord.z;
viewspace position is then viewDirection * pixelDepth