# Ray from screen coord (in worldspace) to farplane

## Recommended Posts

WiredCat    1451

So basically i have a near plane in distance 2.0 and far plane in distance 28.0 * 1000.0

Now when i click a screen i want to make a ray from mouse position to far plane  (perspective projection)

i just tested 4 ways of doing that, none worked  i am not sure wheres bug maybei shouldn't apply aspect ratio to screen height when calculating the size of near and fra plane?

so from start: (the idea top view)

HERES ANOTHER IMAGE SEE ctg(B) -<<< this is what i calc

to calc screen_w i use contanget function (with the fov angle divided by 2)

float a = fov / 2.0;
float cotangent = 1.0 / tan( a * imopi ); (since a is in degrees i must convert it by multiplying (pi/180.0) and ctg(x) is 1.0 / tg(x)

float ax = z_near / cotangent;

float screen_w = 2.0*ax; //multiple by two to get plane width

float yratio = 1.0 / aspect; //then use aspect ratio to find the plane height

float screen_h = screen_w * yratio;

//this is from the input (i calculate the percentage of the coord on screen)
float scr_coord_x = float(x) / float(screen_width);
float scr_coord_y = float(y) / float(screen_height);

//now to calculate the
FPP_CAM is a class that stores position of camera, and direction vectors such front, right and up
dir is froward direction
dirX is right direction
dirY is up direction;

vec3 start_pos = (FPP_CAM->pos + dir * z_near) + (-dirX * (screen_w / 2.0)) + (-dirY * (screen_h/2.0)); //basically here i am going to center of near plane and then go to the bottom left corner

ray.start = start_pos + (dirX * (screen_w * scr_coord_x)) + (dirY * (screen_h * scr_coord_y)); //then after calcing the (near plane in this case dimensions) i scale direction (right and up) by coordinate on the screen multiplied by corresponding near plane dimension.

do the analog for far plane

seems that only when i hit center of the screen i see proper result....
Edited by WiredCat

##### Share on other sites
HappyCoder    5053
The best way to do this is to use your view and projection matrix. That way it doesn't matter if you are using a orthogonal or perspective projection.

If you transform you vertices like this.

clippingSpace = (viewMat * projectionMat) * worldPos
normalizedSpace = clippingSpace / clippingSpace.w

Then you transform points to the world using this equation
worldSpace = inverse(viewMat * projectionMat) * normalizedSpace
worldSpace /= worldSpace.w

However, you can transpose planes from normalizedSpace to worldSpace much easier
// To transform a plane, you multiply it as a 4d vector by the inverse transpose of the matrix
worldPlane = transpose(inverse(inverse(viewMat * projectionMat))) * Vector4(plane.normal.x, plane.normal.y, plane.normal.z, plane.distance)

// however, since you have double inverse you can simplify it to this
worldPlane = transpose(viewMat * projectionMat) * Vector4(plane.normal.x, plane.normal.y, plane.normal.z, plane.distance)

So to generate a ray you can transform two planes, one vertical one hoizontal forming a crosshair at the point on your screen. You then transform those planes to world space using the above code then find the cross product of the normals from the two world space planes to find the direction of the ray. The origin of the ray is the camera position.

To create the two planes you simply do the following

verticalPlane.normal = Vector3(1, 0, 0)
verticalPlane.distance = -(screenPos.x * 2) / screenWidth + 1;

horizontalPlane.normal = Vector3(0, 1, 0)
horizontalPlane.distance = (screenPos.y * 2) / screenHeight + 1;


##### Share on other sites
WiredCat    1451

I am not sure how i can apply that to a custom ray.

i just dont see how can i get ray start and ray end points from such thing

##### Share on other sites
Dirk Gregorius    2757

I always used gluProject() with WinZ = 0 and WinZ =1 for this. This gives you two points on the near and far plane and from there you can build your ray easily. Googling for this resulted in these two links.

https://www.opengl.org/sdk/docs/man2/xhtml/gluUnProject.xml

http://myweb.lmu.edu/dondi/share/cg/unproject-explained.pdf

Sadly this might not work anymore with modern OpenGL so I need to implement this myself soon I guess.

##### Share on other sites
HappyCoder    5053
A ray is simply a start point and a direction, you want to to transform a line segment then you can simply transform two endpoints

using this code
worldSpace = inverse(viewMat * projectionMat) * normalizedSpace
worldSpace /= worldSpace.w

If you are going to be transforming a lot of points then I would only calculate the inverse once, because it is expensive, and reuse it for all the points.

The two points you want to transform in normalizeSpace you can make using

x = (screenPos.x * 2) / screenWidth + 1;
y = -(screenPos.y * 2) / screenHeight + 1;

normalizedA = Vector3(x, y, -1) // replace the -1 with 0 for directx
normalizedB = Vector3(x, y, 1)



##### Share on other sites
WiredCat    1451

Dirk Gregorius   Members   -  Reputation: 1708

Posted Today, 07:21 PM

I always used gluProject() with WinZ = 0 and WinZ =1 for this. This gives you two points on the near and far plane and from there you can build your ray easily. Googling for this resulted in these two links.

I tried to use code from:

https://www.opengl.org/wiki/GluProject_and_gluUnProject_code

but it returns 0,0,0 point (sometimes does not but mostly 0,0,0)

when calling

glhUnProjectf(x, SCREEN_HEIGHT - y, 1.0, modelview.m, projection.m, vp, &ray.end);

please note that i use DirectX matrix order not opengl one( even when i am using opengl) because the standard math is used for that order and opengl uses transposed matrices no wonder why.

Now

that seems that for dx its

worldSpace = inverse(viewMat * projectionMat) * normalizedSpace

and for opengl it would be

worldSpace = inverse(projectionMat * viewMat) * normalizedSpace

i obviously use that first one

so i think

normalizedA = Vector3(x, y, -1) // replace the -1 with 0 for directx ill use 0 there anyway

but still i miss the thing about  converting

normalizedA and normalizedB to normalizedSpace

sorry to be sun an asshole but HappyCodin' i really dont get that way of thinking :X

Edited by WiredCat

##### Share on other sites
Dirk Gregorius    2757

http://richardssoftware.net/Home/Post/23

https://www.mvps.org/directx/articles/improved_ray_picking.htm

This is a problem that must have been solved may times. If you don't want to derive this as an exercise you should be able to find functional code for this in minutes using a search engine of your choice. It seems that there is even an SDK sample.

Edited by Dirk Gregorius

##### Share on other sites
WiredCat    1451

im trying to convert that c# code to c++ but theres a problem i cant get rid of

var ray = new Ray(new Vector3(), new Vector3(vx, vy, 1.0f));
var v = View;
var invView = Matrix.Invert(v);

var toWorld = invView;

ray = new Ray(Vector3.TransformCoordinate(ray.Position, toWorld),

Vector3.TransformCoordinate(ray.Position, toWorld) ->> ray position isnt even set to anything so i consider its 0,0,0 point so theres no use of doing that yet guy does that so i am confused  since constructor like var ray = new Ray(new Vector3(), new Vector3(vx, vy, 1.0f)); (i mean new Vector3() isnt even defined in slimDX specs)

theres only

Icon Member Description Vector3(Single, Single, Single)

Initializes a new instance of the Vector3 class.
Initializes a new instance of the Vector3 class.
Initializes a new instance of the Vector3 class.

that doesnt even make any sense...

but more over i am more confised why my solution doesn't want to work

Edited by WiredCat

##### Share on other sites
Dirk Gregorius    2757

Use new Vector3( 0, 0, 0 ) if you want to a zero vector

##### Share on other sites
WiredCat    1451

i think you got me wrong:

    float vx = (2.0f * float(x) / float(sw) - 1.0f) / ACTUAL_PROJECTION.m[0];
float vy = (-2.0f * float(y) / float(sh) + 1.0f) / ACTUAL_PROJECTION.m[5];

vec3 direction = vec3(vx, vy, 1.0f); //sorry but such direction NOPE
mat4 invView = ACTUAL_VIEW;
invView.Inverse();

mat4 toWorld = invView;
vec3 rstart = invView * vec3(0.0, 0.0, 0.0);
vec3 rdir = invView * direction;
//ray = new Ray(Vector3.TransformCoordinate(ray.Position, toWorld), Vector3.TransformNormal(ray.Direction, toWorld));

rdir = Normalize(rdir);//    ray.Direction.Normalize();

ray_vb[0].v = rstart; // NOPE
ray_vb[1].v = rstart + direction*5000.0;


vec3 rstart = invView * vec3(0.0, 0.0, 0.0); <-- this is hopeless  i will never get anything from that so //Vector3.TransformCoordinate(ray.Position, toWorld) looks like the most useless code in the world

anyway even direction is wrong.

Edited by WiredCat

##### Share on other sites
WiredCat    1451

I decided to post a video of what i get now:

You will see there that sometimes ray isn't computed properly (most of times when reaching corners of the screen)

And full code:


TRay RayFromScreen(int x, int y, int sw, int sh, float fov, float z_near, float z_far, float aspect)
{
mat4 mvm = CAM_MODEL * CAM_VIEW;
mvm.Transpose(); <- i have a code that gets up and right vector from oglmatrix modelview so i just transpose it here
vec3 dirX, dirY;
dirX.x = mvm.m[0];
dirX.y = mvm.m[4];
dirX.z = mvm.m[8];
//
//
dirY.x =	mvm.m[1];
dirY.y =	mvm.m[5];
dirY.z =	mvm.m[9];

TRay res;
//	cot(x) = 1/tan(x)
//	x = z_near / ctg(a);

float a = fov / 2.0;
float cotangent = 1.0 / tan( a * imopi );

float ax = z_near / cotangent;

float screen_w = 2.0*ax;

float yratio = 1.0 / aspect;

float screen_h = screen_w * yratio;

float scr_coord_x = float(x) / float(sw);
float scr_coord_y = float(sh - y) / float(sh);

vec3 dir = FPP_CAM->ReturnFrontVector();

//move to lower left corner
vec3 start_pos = (FPP_CAM->pos + dir * z_near) + (-dirX * (screen_w / 2.0)) + (-dirY * (screen_h/2.0));

res.start = start_pos + (dirX * (screen_w * scr_coord_x)) + (dirY * (screen_h * scr_coord_y));

//compute the world position on the other end since its a perspective projection
ax = z_far / cotangent;
screen_w = 2.0*ax;
screen_h = screen_w * yratio;

start_pos = (FPP_CAM->pos + dir * z_far) + (-dirX * (screen_w / 2.0)) + (-dirY * (screen_h/2.0));

res.end = start_pos + (dirX * (screen_w * scr_coord_x)) + (dirY * (screen_h * scr_coord_y));

return res;
}

//this function calls the frist one
t3dpoint<int> CellFromScreenCPU(int x, int y, int sw, int sh, float fov, float z_near, float z_far, float aspect)
{
TRay ray = RayFromScreen(x, y, sw, sh, fov, z_near, z_far, aspect);
}

actual call:
t3dpoint<int> cell =	CellFromScreenCPU(x,y, SCREEN_WIDTH, SCREEN_HEIGHT, 90.0, 2.0, 28.0 * 1000.0, float(SCREEN_WIDTH) / float(SCREEN_HEIGHT));

Edited by WiredCat

##### Share on other sites
IYP    1545

lets say you have clicked on a point at (x,y) first you need to normalize it, which will be (x/width, y/height) next thing will be no move this to NDC coordinates, this point is considered on the near plane so what you do is:

(you need farZ, nearZ, camera position and projection and view matrices)

(I have used this method it works perfect for me)

vec2 NormalizedCoordinates=ScreenCoordinates/ScreenSize;
vec4 NDC=vec4( (NormalizedCoordinates-0.5)*2,-1,1); //z=-1 because its on near plane

TRay res;

res.start=vec3(inverse(ProjectionMatrix*ViewMatrix)*NDC);//this move our point from the NDC coordinates to world coordinates

vec3 RayDirection=vec3(res.start-CameraPosition);//ray direction is from camera to ray's starting point
RayDirection=RayDirection/RayDirection.z*(farZ-nearZ);//normalize by Z resize the z by farZ-nearZ so that it hits the far plane

res.end=res.start+RayDirection;

##### Share on other sites
WiredCat    1451

nah it doesnt work btw i think NormalizedCoordinates - 0.5 is wrong since to get ndc coord you need to do * 2.0 - 1.0

any ways function always shoots ray from the screen center to the 0,0,0 point...(or to be more precise from 0,0,0 point to direction(from 0,0,0 to cam_pos)*z_far...  no matter what i set to normalized coords


TRay RayFromScreen(int x, int y, int sw, int sh, float fov, float z_near, float z_far, float aspect)
{

vec2 NormalizedCoordinates;
NormalizedCoordinates.x = (float(x) / float(sw) ) * 2.0 - 1.0;
NormalizedCoordinates.y = (float(y) / float(sh) ) * 2.0 - 1.0;

vec4 NDC=vec4(NormalizedCoordinates.x, NormalizedCoordinates.y,-1.0, 1.0); //z=-1 because its on near plane

TRay res;
mat4 ProjectionMatrix = CAM_PROJECTION;
ProjectionMatrix.Transpose();
mat4 ViewMatrix = CAM_VIEW;
ViewMatrix.Transpose();
mat4 pvm = ProjectionMatrix*ViewMatrix;
pvm.Inverse();
res.start=pvm*NDC;

vec3 RayDirection=res.start-FPP_CAM->pos;
RayDirection=(RayDirection/RayDirection.z)*(z_far-z_near);//normalize by Z resize the z by farZ-nearZ so that it hits the far plane

res.end=res.start+RayDirection;

return res;
}

Edited by WiredCat

##### Share on other sites
IYP    1545

about the NDC cords "vec4( (NormalizedCoordinates-0.5)*2,-1,1);" is the same as vec4( NormalizedCoordinates*2-1.0,-1,1);.

About the function, it seems that your doing the exact thing I said, and if you agree the function mathematically should be working, one thing I suspect is that maybe your projection matrix has a different behavior, esp. on mapping z, i'd say take a look at the res.start and check if it's calculated correctly or if its w is 1, your matrix may result in other values for w so yes that's what I suspect.

##### Share on other sites
WiredCat    1451

the code i wrote is working good but i had to disable the apsect ration for y screen, looks like fov is for screen HEIGHT not for WIDTH, i am confused

and aspect is width / height

	float a = fov / 2.0;
float cotangent = 1.0 / tan( a * imopi );

float ax = z_near / cotangent;

float screen_w = 2.0*ax;

float screen_h = screen_w;// * yratio; <--- now i dont multiple by 1.0/aspect

screen_w = screen_w * aspect;  <--- here

float scr_coord_x = float(x) / float(sw);
float scr_coord_y = float(sh - y) / float(sh);

Edited by WiredCat

##### Share on other sites
Randy Gaul    2762

It sounds like you mostly got it working. If you wanted a reference there's a piece in Essential Math by Van Verth that covers screen point to world space ray. There's also example code that does this on the books CD-ROM.

I think I found their example code:

//-------------------------------------------------------------------------------
// @ Game::GetPickRay()
//-------------------------------------------------------------------------------
// Get pick ray from screen position
//-------------------------------------------------------------------------------
IvVector3
Game::GetPickRay( float sx, float sy, float fov, float width, float height )
{
float d = 1.0f/IvTan(fov*kPI/360.0f);
float aspect = width/height;
IvVector3 viewPoint( 2.0f*aspect*sx/width - aspect, -2.0f*sy/height + 1.0f,
IvRenderer::mRenderer->GetAPI() == kOpenGL ? -d : d );

viewPoint = mViewToWorldMatrix.TransformPoint( viewPoint );

return viewPoint-mEyePoint;
}


Which is used like this:

//-------------------------------------------------------------------------------
// @ Game::Update()
//-------------------------------------------------------------------------------
// Main update loop
//-------------------------------------------------------------------------------
void
Game::UpdateObjects( float /*dt*/ )
{
// handle picking
unsigned int h, v;
if (mEventHandler->IsMouseDown( h, v ))
{
// get the pick ray
IvVector3 ray = GetPickRay( (float) h, (float) v, IvRenderer::mRenderer->GetFOV(),
(float) IvRenderer::mRenderer->GetWidth(), (float) IvRenderer::mRenderer->GetHeight() );

// compute intersection with z=0 plane
float t = -mEyePoint.GetZ()/ray.GetZ();
mClickPosition = mEyePoint + t*ray;

mEventHandler->MouseUp();
}

}   // End of Game::Update()


Ran the demo for this:

##### Share on other sites
IYP    1545

well it depend what the fov is for, it mostly for Y, depend on your projection matrix function

##### Share on other sites
WiredCat    1451

i did something liek this:

		float sx = x;
float sy = y;
float width = sw;
float height= sh;

float d = 1.0f/tan(fov*pi/360.0f);
//  float aspect = width/height;
vec3 viewPoint = vec3( 2.0f*aspect*sx/width - aspect, -2.0f*sy/height + 1.0f, -d);//// : d );
mat4 invV = CAM_VIEW;
invV.Inverse();
vec3 viewPointI = invV * viewPoint; //i assume thats viewToWorldMatrix

res.start 	= FPP_CAM->pos;
res.end 		= res.start + Normalize(viewPointI-FPP_CAM->pos)*4000.0;
return res;


that really doesnt work :/

HELP!

##### Share on other sites
IYP    1545

nah it doesnt work btw i think NormalizedCoordinates - 0.5 is wrong since to get ndc coord you need to do * 2.0 - 1.0

any ways function always shoots ray from the screen center to the 0,0,0 point...(or to be more precise from 0,0,0 point to direction(from 0,0,0 to cam_pos)*z_far...  no matter what i set to normalized coords


TRay RayFromScreen(int x, int y, int sw, int sh, float fov, float z_near, float z_far, float aspect)
{

vec2 NormalizedCoordinates;
NormalizedCoordinates.x = (float(x) / float(sw) ) * 2.0 - 1.0;
NormalizedCoordinates.y = (float(y) / float(sh) ) * 2.0 - 1.0;

vec4 NDC=vec4(NormalizedCoordinates.x, NormalizedCoordinates.y,-1.0, 1.0); //z=-1 because its on near plane

TRay res;
mat4 ProjectionMatrix = CAM_PROJECTION;
ProjectionMatrix.Transpose();
mat4 ViewMatrix = CAM_VIEW;
ViewMatrix.Transpose();
mat4 pvm = ProjectionMatrix*ViewMatrix;
pvm.Inverse();
res.start=pvm*NDC;

vec3 RayDirection=res.start-FPP_CAM->pos;
RayDirection=(RayDirection/RayDirection.z)*(z_far-z_near);//normalize by Z resize the z by farZ-nearZ so that it hits the far plane

res.end=res.start+RayDirection;

return res;
}


the function is correct here, it should be working, though your new function is not, maybe send the code that makes the projection and view matrices, that may help, also I really am not getting what the problem is, maybe some videos or pics help

##### Share on other sites
WiredCat    1451

ray isnt hitting desired part of screen

void Translate(T x, T y, T z)
{
Matrix44<T> tran( 1, 0, 0, x,
0, 1, 0, y,
0, 0, 1, z,
0, 0, 0, 1);
(*this) = (*this) * tran;
}

template <class T> void
glLookAt(Matrix44<T> &matrix, t3dpoint<T> eyePosition3D, t3dpoint<T> center3D, t3dpoint<T> upVector3D )
{
t3dpoint<T>  forward, side, up;
forward = Normalize( vectorAB(eyePosition3D, center3D) );
side = Normalize( forward * upVector3D );
up = side * forward;

matrix.m[0] = side.x;
matrix.m[1] = side.y;
matrix.m[2] = side.z;

matrix.m[4] = up.x;
matrix.m[5] = up.y;
matrix.m[6] = up.z;

matrix.m[8] 	= -forward.x;
matrix.m[9] 	= -forward.y;
matrix.m[10] 	= -forward.z;

Matrix44<T> transgender;
transgender.Translate(-eyePosition3D.x, -eyePosition3D.y, -eyePosition3D.z);

matrix = transgender * matrix;
}

template <class T>
void gluPerspectiveA(Matrix44<T> & matrix, T fovy, T aspect, T zmin, T zmax)
{
T xmin, xmax, ymin, ymax;
ymax = zmin * tan(fovy * M_PI / 360.0);
ymin = -ymax;
xmin = ymin * aspect;
xmax = ymax * aspect;
//mglFrustum(m, xmin, xmax, ymin, ymax, zmin, zmax);
//mglFrustum(Matrix44<T> & matrix, T l, T r, T b, T t, T n, T f)

matrix.m[0] = 	(2.0*zmin)/(xmax-xmin);
matrix.m[1] = 	0.0;
matrix.m[2] =  	(xmax + xmin) / (xmax - xmin);
matrix.m[3] =	0.0;

matrix.m[4] = 	0.0;
matrix.m[5] = 	(2.0*zmin) / (ymax - ymin);
matrix.m[6] =   (ymax + ymin) / (ymax - ymin);
matrix.m[7] =   0.0;

matrix.m[8] = 	0.0;
matrix.m[9] =  	0.0;
matrix.m[10] =	-(zmax + zmin) / (zmax-zmin);
matrix.m[11] =  (-2.0*zmax*zmin) / (zmax-zmin);

matrix.m[12] =  0.0;
matrix.m[13] =  0.0;
matrix.m[14] =  -1.0;
matrix.m[15] =  0.0;

}



##### Share on other sites
IYP    1545

ok I guess I found you're problem:

matrix.m[8] = -forward.x;
matrix.m[9] = -forward.y;
matrix.m[10] = -forward.z;

you have the forward in the opposite direction (this is the right way I do it like this too but references and actual related math don't do like this) so you need to negate the z value of the starting point after calculating it

code will be like:

TRay RayFromScreen(int x, int y, int sw, int sh, float fov, float z_near, float z_far, float aspect)
{

vec2 NormalizedCoordinates;
NormalizedCoordinates.x = (float(x) / float(sw) ) * 2.0 - 1.0;
NormalizedCoordinates.y = (float(y) / float(sh) ) * 2.0 - 1.0;

vec4 NDC=vec4(NormalizedCoordinates.x, NormalizedCoordinates.y,-1.0, 1.0); //z=-1 because its on near plane

TRay res;
mat4 ProjectionMatrix = CAM_PROJECTION;
ProjectionMatrix.Transpose();
mat4 ViewMatrix = CAM_VIEW;
ViewMatrix.Transpose();
mat4 pvm = ProjectionMatrix*ViewMatrix;
pvm.Inverse();
res.start=pvm*NDC;
res.start.z=res.start.z*-1.0f;

vec3 RayDirection=res.start-FPP_CAM->pos;
RayDirection=(RayDirection/RayDirection.z)*(z_far-z_near);//normalize by Z resize the z by farZ-nearZ so that it hits the far plane

res.end=res.start+RayDirection;

return


##### Share on other sites
WiredCat    1451

gluperspective function accepts fov as fovy not fov for xz plane

##### Share on other sites
IYP    1545

I'm not getting your point, maybe chat'd be better

##### Share on other sites
WiredCat    1451

for gluPerspecitve you define a fov angle which is between top and bottom clipplanes,

##### Share on other sites
IYP    1545

yes I understand that, but where are you getting with that? it's supposed to be like that in other functions too, you have fovy, D3D does the same too

## Create an account

Register a new account