Direct3D Screen Coordinates / Projection
Author
I tell you, patience is needed to lean this 3D stuff. Mine has certainly been tested recently. Anyway, on to my next problem, which again sounds simple but I can''t figure it out (too thick I guess). I''m using Direct3D immediate mode and I want to take a 2D screen position and convert to a 3D point to find the nearest vertex or object (yes, i''m attempting to write an edtor). My problem lies in not really understanding how perspective transformations work, and how Direct3D converts 3D->2D. I can convert each vertex point into camera space by multiplying it by the view matrix. But how do I use the projection matrix to get a screen position? Multiplying it does not seem to work. Are there any tutorials on this on the web? I can''t seem to find much. The 3D tutorials out there simply take x and y and divide by z in order to get depth on the screen. This I understand fine.
BTW i''m using Borland C++Builder and can''t use the D3DX library as Microsoft (bless em!) did not supply a Borland LIB for this extra library.
Thanks again. Your help much appreciated.
To convert from 3D to 2D you just multiply with the projection matrix. But what you might miss is that you have to convert back from homogenous coordinates to normal 3D coordinates by dividing x,y, and z with w.
D3DMATRIX M;
D3DVECTOR V;
float x,y,z,w;
x = V.x*M._11 + V.y*M._21 + V.z*M._31 + M._41;
y = V.x*M._12 + V.y*M._22 + V.z*M._32 + M._42;
z = V.x*M._13 + V.y*M._23 + V.z*M._33 + M._43;
w = V.x*M._14 + V.y*M._24 + V.z*M._34 + M._44;
V.x = x/w;
V.y = y/w;
V.z = z/w;
Anyway, if you know the field of view for your projection it is not that hard to unproject your vertices.
Start by computing how the width and height of the view frustum at a fixed distance (let''s use distance = 1).
float Width = 2*cos(FovX/2);
float Height = Width*ScreenHeight/ScreenWidth;
Then scale the screen coordinates so that ScreenWidth and ScreenHeight fit inside the view frustum.
D3DVECTOR V;
V.x = (ScreenX-ScreenWidth/2) * Width/ScreenWidth;
V.y = (ScreenY-ScreenHeight/2) * Height/ScreenHeight;
V.z = 1; // The same distance we chose before
And there you have it, now you need to transform this vector from view space to world space in order to find which objects the ray, going through the camera and this vector, intersects.
I hope that gave you some insight into how projection works.
- WitchLord
D3DMATRIX M;
D3DVECTOR V;
float x,y,z,w;
x = V.x*M._11 + V.y*M._21 + V.z*M._31 + M._41;
y = V.x*M._12 + V.y*M._22 + V.z*M._32 + M._42;
z = V.x*M._13 + V.y*M._23 + V.z*M._33 + M._43;
w = V.x*M._14 + V.y*M._24 + V.z*M._34 + M._44;
V.x = x/w;
V.y = y/w;
V.z = z/w;
Anyway, if you know the field of view for your projection it is not that hard to unproject your vertices.
Start by computing how the width and height of the view frustum at a fixed distance (let''s use distance = 1).
float Width = 2*cos(FovX/2);
float Height = Width*ScreenHeight/ScreenWidth;
Then scale the screen coordinates so that ScreenWidth and ScreenHeight fit inside the view frustum.
D3DVECTOR V;
V.x = (ScreenX-ScreenWidth/2) * Width/ScreenWidth;
V.y = (ScreenY-ScreenHeight/2) * Height/ScreenHeight;
V.z = 1; // The same distance we chose before
And there you have it, now you need to transform this vector from view space to world space in order to find which objects the ray, going through the camera and this vector, intersects.
I hope that gave you some insight into how projection works.
- WitchLord
Author
Thanks for the reply but I am not having much success getting it to work. As I understand the first part of your code....
D3DMATRIX M;D3DVECTOR V;
float x,y,z,w;
x = V.x*M._11 + V.y*M._21 + V.z*M._31 + M._41;
y = V.x*M._12 + V.y*M._22 + V.z*M._32 + M._42;
z = V.x*M._13 + V.y*M._23 + V.z*M._33 + M._43;
w = V.x*M._14 + V.y*M._24 + V.z*M._34 + M._44;
V.x = x/w;
V.y = y/w;
V.z = z/w;
takes a vector which I assume is in camera space, and calculates V as a screen coordinate. I start by setting up a vector at 0,0,0. I then multiply it by the camera matrix (which is at 0, 1, 5 looking at 0, 1, 0) to transform it to camera space. I assumed the above code would return me a screen offset which I add to the centre of the viewport, but it does not seem to work the way I thought it would. To test this I am drawing a circle at the returned screen position overlaying the 3D rendered scene. The circle appears in the middle of the viewport sure enough but when I rotate the camera away from 0, 0, 0 the circle stays in the middle of the screen. I am assuming the the above code converts 3D->2D and the other part 2D-3D. Thanks for your input. If you can help me further I would much appreciate it.
Regards.
John.
D3DMATRIX M;D3DVECTOR V;
float x,y,z,w;
x = V.x*M._11 + V.y*M._21 + V.z*M._31 + M._41;
y = V.x*M._12 + V.y*M._22 + V.z*M._32 + M._42;
z = V.x*M._13 + V.y*M._23 + V.z*M._33 + M._43;
w = V.x*M._14 + V.y*M._24 + V.z*M._34 + M._44;
V.x = x/w;
V.y = y/w;
V.z = z/w;
takes a vector which I assume is in camera space, and calculates V as a screen coordinate. I start by setting up a vector at 0,0,0. I then multiply it by the camera matrix (which is at 0, 1, 5 looking at 0, 1, 0) to transform it to camera space. I assumed the above code would return me a screen offset which I add to the centre of the viewport, but it does not seem to work the way I thought it would. To test this I am drawing a circle at the returned screen position overlaying the 3D rendered scene. The circle appears in the middle of the viewport sure enough but when I rotate the camera away from 0, 0, 0 the circle stays in the middle of the screen. I am assuming the the above code converts 3D->2D and the other part 2D-3D. Thanks for your input. If you can help me further I would much appreciate it.
Regards.
John.
The first part of my post showed you how the vertices are multiplied with the matrices. I did not write how to set up the projection matrix as I assumed you knew how to do that.
Let me take a quick run down the geometry pipeline and tell you the steps required to get from model space to screen space:
All these conversions are described with matrices. If the matrices are concatenated into one by multiplying them together the conversion from model space to screen coordinates can be performed by one matrixmultiplication only. When you concatenate the matrices you must be careful to do it in the right order, otherwise your calculations will be screwed up.
ScreenVertex = Vertex * WorldMtx * ViewMtx * ProjMtx * ViewportMtx;
After conversion to normal 3D vector, you have the screen coordinate in ScreenVertex.x and ScreenVertex.y. ScreenVertex.z is used in depth test with the z-buffer.
---
If you still can't get it to work then you'll have to post some code to show how you are doing your calculations.
- WitchLord
Edited by - WitchLord on 4/24/00 1:53:22 PM
Let me take a quick run down the geometry pipeline and tell you the steps required to get from model space to screen space:
- The vertex is initialized in model space. For example a vertex from you monster model.
- The vertex is converted from model space to world space by multiplying with the world matrix.
- The vertex is converted from world space into view space with the view matrix. (View space and camera space is just two names for the same thing.)
- The vertex is projected into normalized screen coordinates with the projection matrix. If the vertex is inside the bounding box ranging from (-1,-1,0) to (1,1,1) then it is inside the view frustum and visible.
- The vertex is finally scaled and translated to the actual screen coordinates. This step is described by setting the viewport.
All these conversions are described with matrices. If the matrices are concatenated into one by multiplying them together the conversion from model space to screen coordinates can be performed by one matrixmultiplication only. When you concatenate the matrices you must be careful to do it in the right order, otherwise your calculations will be screwed up.
ScreenVertex = Vertex * WorldMtx * ViewMtx * ProjMtx * ViewportMtx;
After conversion to normal 3D vector, you have the screen coordinate in ScreenVertex.x and ScreenVertex.y. ScreenVertex.z is used in depth test with the z-buffer.
---
If you still can't get it to work then you'll have to post some code to show how you are doing your calculations.
- WitchLord
Edited by - WitchLord on 4/24/00 1:53:22 PM
Author
Thanks for the response, WitchLord, and for staying with me on this. I think I understand what you are saying but I just can't get Direct3D to play the game. Here's my code.
//First I get the view matrix and multiply it with the
//projection matrix
D3DMATRIX M;
d3device->GetTransform(D3DTRANSFORMSTATE_VIEW, &M);
d3device->MultiplyTransform(D3DTRANSFORMSTATE_PROJECTION, &M);
//I set my vector to be 0,0,0 in world space. My camera is
//set back from this looking at this point so it is
//definantly visible
D3DVECTOR V = D3DVECTOR(0, 0, 0);
//I then apply the view/projection matrix with the vector
//float x,y,z,w;
x = V.x*M._11 + V.y*M._21 + V.z*M._31 + M._41;
y = V.x*M._12 + V.y*M._22 + V.z*M._32 + M._42;
z = V.x*M._13 + V.y*M._23 + V.z*M._33 + M._43;
w = V.x*M._14 + V.y*M._24 + V.z*M._34 + M._44;
What I would expect to see (from what you describe) in x, y, z, w is a normalized screen coordinate from -1, -1,0 to 1,1,1. But what I get is 0, -1, 5, w=1. Any ideas?
BTW. I've been to your webpage. very nice.
Edited by - johnp on 4/25/00 3:37:36 PM
//First I get the view matrix and multiply it with the
//projection matrix
D3DMATRIX M;
d3device->GetTransform(D3DTRANSFORMSTATE_VIEW, &M);
d3device->MultiplyTransform(D3DTRANSFORMSTATE_PROJECTION, &M);
//I set my vector to be 0,0,0 in world space. My camera is
//set back from this looking at this point so it is
//definantly visible
D3DVECTOR V = D3DVECTOR(0, 0, 0);
//I then apply the view/projection matrix with the vector
//float x,y,z,w;
x = V.x*M._11 + V.y*M._21 + V.z*M._31 + M._41;
y = V.x*M._12 + V.y*M._22 + V.z*M._32 + M._42;
z = V.x*M._13 + V.y*M._23 + V.z*M._33 + M._43;
w = V.x*M._14 + V.y*M._24 + V.z*M._34 + M._44;
What I would expect to see (from what you describe) in x, y, z, w is a normalized screen coordinate from -1, -1,0 to 1,1,1. But what I get is 0, -1, 5, w=1. Any ideas?
BTW. I've been to your webpage. very nice.
Edited by - johnp on 4/25/00 3:37:36 PM
OK. From what I see in this post you have a fair understanding of the 3D math. What you are missing is the meaning of MultiplyTransform(). This function multiplied the transform matrix (in this case the projection matrix) with the supplied matrix. It does not update your supplied matrix. So when you call this function in your program you are in fact screwing up the DirectX geometry pipeline by destroying its projection matrix. What you should do is this:
D3DMATRIX M, M2;
d3device->GetTransform(D3DTRANSFORMSTATE_VIEW, &M);
d3device->GetTransform((D3DTRANSFORMSTATE_PROJECTION, &M2);
D3DMath_MatrixMultiply(M, M, M2); // This function exist in the framework that comes with the SDK
Then you can continue as before. After multiplying the vector with the matrix, the vectors coordinates should be described in normalized screen coordinates (it can still have values outside the box though, all that means is that the vertex is outside the viewfrustum).
Thanks, I''m glad you liked my homepage. If I have the time I will most likely expand it with Direct3D tutorials (like NeHe''s OpenGL tutorials
). But I have not made any final decisions on this.
- WitchLord
D3DMATRIX M, M2;
d3device->GetTransform(D3DTRANSFORMSTATE_VIEW, &M);
d3device->GetTransform((D3DTRANSFORMSTATE_PROJECTION, &M2);
D3DMath_MatrixMultiply(M, M, M2); // This function exist in the framework that comes with the SDK
Then you can continue as before. After multiplying the vector with the matrix, the vectors coordinates should be described in normalized screen coordinates (it can still have values outside the box though, all that means is that the vertex is outside the viewfrustum).
Thanks, I''m glad you liked my homepage. If I have the time I will most likely expand it with Direct3D tutorials (like NeHe''s OpenGL tutorials
). But I have not made any final decisions on this.- WitchLord
Author
doh! now it makes sense but i''m still not quite there yet. When I multiply the view/projection matrix with my vector I get a an X value that looks okay but Y is 1.9999999 which is outside the frustrum. Here''s the code.
//This value represents a corner of cube which is in the middle of the screen and is visible.
D3DVECTOR V = D3DVECTOR(-1, 2, 0);
x = V.x*M._11 + V.y*M._21 + V.z*M._31 + M._41;
y = V.x*M._12 + V.y*M._22 + V.z*M._32 + M._42;
z = V.x*M._13 + V.y*M._23 + V.z*M._33 + M._43;
w = V.x*M._14 + V.y*M._24 + V.z*M._34 + M._44;
//i''m not sure why w = 0 sometimes, but without this a get
//a FP error. Also, with a vector of 0,0,0 (worldspace)
//I get a result of 0,0,0.
if (w == 0)
{
w = 1;
}
V.x = x/w;
V.y = y/w;
V.z = z/w;
D3DVIEWPORT7 vp;
device->GetViewport(&vp);
float scalex = vp.dwWidth / 2;
float scaley = vp.dwHeight / 2;
int screenx = V.x * scalex;
int screeny = V.y * scaley;
If a draw a small square over the 3D scene at this point it now moves when I rotate the camera, but only horizontally. It always appears at the bottom of the screen. Thanks again for your help.
//This value represents a corner of cube which is in the middle of the screen and is visible.
D3DVECTOR V = D3DVECTOR(-1, 2, 0);
x = V.x*M._11 + V.y*M._21 + V.z*M._31 + M._41;
y = V.x*M._12 + V.y*M._22 + V.z*M._32 + M._42;
z = V.x*M._13 + V.y*M._23 + V.z*M._33 + M._43;
w = V.x*M._14 + V.y*M._24 + V.z*M._34 + M._44;
//i''m not sure why w = 0 sometimes, but without this a get
//a FP error. Also, with a vector of 0,0,0 (worldspace)
//I get a result of 0,0,0.
if (w == 0)
{
w = 1;
}
V.x = x/w;
V.y = y/w;
V.z = z/w;
D3DVIEWPORT7 vp;
device->GetViewport(&vp);
float scalex = vp.dwWidth / 2;
float scaley = vp.dwHeight / 2;
int screenx = V.x * scalex;
int screeny = V.y * scaley;
If a draw a small square over the 3D scene at this point it now moves when I rotate the camera, but only horizontally. It always appears at the bottom of the screen. Thanks again for your help.
When you transform from the normalize screen space you should do it like this:
x = x * vp.dwWidth/2 + vp.dwX + vp.dwWidth/2;
y = y * -vp.dwHeight/2 + vp.dwY + vp.dwHeight/2;
The minus in the y coordinate is because the screen''s y axis is pointing down while the normalize screen space''s y axis is pointing up. I also add vp.dwX + vp.dwWidth/2 to center the view in the viewport and then translate with the viewports top-left position.
Other than that I can''t help you. You say that Direct3D does the correct transformation, i.e. the box shows up where it is supposed to be? Then the viewport problem must be your last problem, unless you are doing something with the worldmatrix as well.
Cheers
- WitchLord
x = x * vp.dwWidth/2 + vp.dwX + vp.dwWidth/2;
y = y * -vp.dwHeight/2 + vp.dwY + vp.dwHeight/2;
The minus in the y coordinate is because the screen''s y axis is pointing down while the normalize screen space''s y axis is pointing up. I also add vp.dwX + vp.dwWidth/2 to center the view in the viewport and then translate with the viewports top-left position.
Other than that I can''t help you. You say that Direct3D does the correct transformation, i.e. the box shows up where it is supposed to be? Then the viewport problem must be your last problem, unless you are doing something with the worldmatrix as well.
Cheers
- WitchLord
Author
At last!!! It works. The problem I was having was to do with the matrix multiplication. I had copied the D3DUtil_MatrixMultiply code into my own Matrix class. Anyway, it obviously does not work because using the D3DUtil code works fine. Now to go the other way 2D->3D!!!!!
Thanks again for your help. I see your name a lot in post replies. It is really kind of you to make the effort and respond to these problems. It is not always possible to learn from a book, and some people (like me) take a while before they "get it".
Thanks again for your help. I see your name a lot in post replies. It is really kind of you to make the effort and respond to these problems. It is not always possible to learn from a book, and some people (like me) take a while before they "get it".
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
