My question is, how do i project(unproject?) screen-space position back to view space?
object space -> world transform -> world space -> view transform -> camera space -> projection transform -> screen space.
do you mean back to camera space, or world space?
given a 2d point on the screen. you can cast a ray into the scene and determine intersections in world space.
without z information, which is usually discarded after the divide by z foreshortening, there's no way to back-solve (un-project) from screen space back to camera space. going from camera space back to world space is trivial: perform your camera (view) transform in reverse order: move(-cam_x, -cam_y, -cam_z). then rotate(-cam_zrot), then rotate(-cam_yrot) then rotate(-cam_xrot). this assumes an x,y,z rotation order is used in the game.
if you DO have z info, simply reverse the mathematical process used to transform your camera space vertex to screen coordinates and a left over z value. this would be the projection transform, and the translation to screen coordinates, applied in reverse order. IE convert your screen pixel and leftover z value back to a (homogeneous?) projection in the range -1,-1 to 1,1 (undo the screen coordinate translation). then apply the reverse of the math in the projection matrix (un-project). you can find the formulas for both in the directx docs, Ogl surely has similar formulas in their docs.
i don't know if the left over z values can be extracted from directx/Ogl or not. you could always apply a transform matrix on a vertex and use the result to get your z, but there you're reproducing the math that directx is already doing when drawing.
the brute force approach would be to put your vertex in a v3, then use v3transform() [i think thats the name of it] with the world, view, and projection mats. this would give you the x,y,z coords of the vertex in object space (before you start), world space (after world transform), and camera space (after view transform), as well as the 2d screen coordinates of the vertex (after projection transform). then you just use the values from the section of the pipeline you want. you could even create a projection matrix that doesn't include the screen translation, just the projection, and get the x,y,z coords for homogeneous screen space (after projection, but before translation to screen coords). but again this is solving for things that the graphics pipeline is already solving for (duplication of effort).
Edited by Norman Barrows, 13 July 2013 - 09:45 AM.
"DirectX is like a belt fed machine gun, where every texture change is like hand loading in a new belt of ammo. worse, every mesh (vb) is a new belt of ammo, and a texture is like breaking the gun down, and setting it up again elsewhere, then loading it, then spraying triangles again. so you want to setup the gun once, string all your belts together, load it once, then just spray."
Rockland Software Productions
"Building PC games since 1988"