Let x and y be the point on the screen which we want to convert to the world space.
We need to create a ray which passes through the camera and the point(x, y) in worldSpace
This ray needs to satisfy the line equation [eqn]Pl = C + t * D [/eqn] (1)
where Pl = point along the line.
C = Point on the line. In this case the camera position
D = the Direction of the ray
t = some float
Now we need to get the direction of the ray.
First we transform the point to our View space coordinates. View space ranges from -1 to 1 on the x and y axis and 0 to 1 on the z axis with (0,0) being the center of the screen. We can set viewSpaceZ to 1 since the ray goes into the screen
We just reverse the following formula which is used to scale the the position in the world to fit the viewport size.
x = (ViewSpaceX + 1) * Viewport.Width * 0.5 + Viewport.TopLeftX
y = (1 - ViewSpaceY) * Viewport.Height * 0.5 + Viewport.TopLeftY
z = Viewport.MinDepth + ViewSpaceZ * (Viewport.MaxDepth - Viewport.MinDepth)
[spoiler]
viewSpaceX = ((2.0f * x) / screenWidth) - 1.0f;
viewSpaceY = -((2.0f * y) / screenHeight - 1.0f);
viewSpaceZ = 1.0f;
[/spoiler]
Next we adjust the points using the projection matrix to account for the aspect ratio of the viewport.
[spoiler]
viewSpaceX = viewSpaceX / projectionMatrix._11;
viewSpaceY = viewSpaceY/ projectionMatrix._22;
viewSpaceZ = viewSpaceY/ projectionMatrix._33;
[/spoiler]
To get our world matrix we just need to invert our view matrix.
Once we have our world matrix we can get the direction of our ray by transforming the viewspace point with the world matrix and then normalize the resultant vector.
[spoiler]
directionX = (viewSpaceX * worldMatrix._11) + (viewSpaceY * worldMatrix._21) + (viewSpaceZ * worldMatrix._31);
directionY = (viewSpaceX * worldMatrix._12) + (viewSpaceY * worldMatrix._22) + (viewSpaceZ * worldMatrix._32);
directionZ = (viewSpaceX * worldMatrix._13) + (viewSpaceY * worldMatrix._23) + (viewSpaceZ * worldMatrix._33);
// Normalize direction vector
[/spoiler]
At this stage, we have an infinite number of positions in the world which will fall on this ray(depending on the value of t)
To get the exact point in the world we need the intersection of this ray with the viewing plane.
The viewing plane be defined by the plane equation [eqn]Ax + By + Cz + k = 0[/eqn]
where N = (A, B, C) = the plane normal vector
k = Distance from the origin
Alternatively, the plane equation can be written as [eqn]dot(Pp,N) = k[/eqn] (2)
where Pp is the point on plane
To get the normal of our viewing plane in world space, we need the combination matrix of our view and projection matrix
[eqn]M = V.P. [/eqn]
Once we have our matrix M, the viewing plane normal is simply the normalized vector of (M._13, M._23, M._33)
When the plane and line intersect , [eqn]Pl = Pp[/eqn]
Substituting (1) in 2, we get
[eqn]dot(C + t *d, N) = k[/eqn]
Using distribution, we get
[eqn]dot(C + t *d, N) = dot(C,N) + t*dot(D,N) = k[/eqn]
Solving for t, we get
[eqn]t = (k - dot(C,N) ) / dot (D,N)[/eqn]
Once we have t, we can plug it back in equation (1) to get the point in our world.
This ends this mini tutorial of converting our screen coordinates to world coordinates
