Well... shouldn't this be that simple:
object space ----[world a.k.a. model matrix]----> world space
world space ----[view a.k.a. camera matrix]----> view space
view space ----[projection matrix]----> clip space
object space <----[inverse world a.k.a. model matrix]---- world space
world space <----[inverse view a.k.a. camera matrix]---- view space
view space <----[inverse projection matrix]---- clip space
Anyways, this is how it *seems right* to me, but I'm not a guru here. Maybe I'm being picky ;) about naming and that was not the intention of this topic (but still I wanted to clarify naming before I ask my question(s) and make more confusion).
So, the reason I post is because (obviously) I have a problem with picking. The issue here is that in my renderer I use right hand coordinate system, like in OpenGL (for sake of compatibility, I have OGL renderer in this app too and I don't want to negate every needed value to get the same result, it would only make more future errors).
So I construct my projection matrix using D3DXMatrixPerspectiveFovRH() and view matrix using D3DXMatrixLookAtRH(). Before sending them to HLSL I transpose them (for some reason I have to do this, otherwise I get incorrect results [DX stores matrices in row major, but in HLSL they need to be in column major?]). All is sweet and dandy until picking occurs. I'm pretty sure that I'm doing something wrong, because this is my first attempt with renderer independent picking. I follow what's in the tutorial but intersection test gives incorrect results. For sake of simplicity my sphere is at (0,0,0) so I don't have to care about world and invWorld matrices. I'm guessing that something is wrong with my matrices but it's hard to track down.
Also I'm not sure what's going on here (tutorial):
// Adjust the points using the projection matrix to account for the aspect ratio of the viewport.
pointX = pointX / projectionMatrix._11;
pointY = pointY / projectionMatrix._22;
and how exactly the unprojecting part works. I mean I have mouse coordinates that I rescale into -1, 1 range but how do I get from vec2 to vec3? Where does the 3rd component come from?