# OpenGL Picking using raycasting from the camera

This topic is 3477 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

Hi, I'm trying to implement picking in OpenGL using raycasting from the camera (since the method is fairly API-indipendant I didn't put this in the OpenGL section). The app picks up collisions with axis-aligned cubes fine, but behaves incorrectly with rotated cubes. My method to intersect them is to translate them to the origin and set their rotations to 0 (ie make them axis-aligned). I then displace the ray start/end by the distance I had to move the cube, and rotate the ray start/end about the origin by the negative of the amount I had to rotate the cube. In code it looks something like this:
	bool cLineSegment::CheckForCollision(const cObjectAlignedBoundingBox& oabb,
cCollisionReport& report)const
{
Vector3 translation = oabb.mPosition;

//Get the rotation about each axis
float alpha = oabb.mOrientation.x;
float beta = oabb.mOrientation.y;
float gamma = oabb.mOrientation.z;

cLineSegment rotatedLine = *this;
//Translate to origin
rotatedLine.mp0 -= translation;
rotatedLine.mp1 -= translation;
rotatedLine.mp0 = RotateByOrigin(rotatedLine.mp0,-alpha,-beta,-gamma);
rotatedLine.mp1 = RotateByOrigin(rotatedLine.mp1,-alpha,-beta,-gamma);

//								position							extents
cAxisAlignedBoundingBox axisBox(MyMaths::Vector3(0.0f,0.0f,0.0f),oabb.mExtents);

if(rotatedLine.CheckForCollision(axisBox, report))
{
//Rotate stuff back to get accurate world coordiantes of collision
// <here>
return true;
}

return false;
}


	Vector3 RotateByOrigin(const Vector3& currentPos, const float& alpha, const float& beta, const float& gamma)
{
Vector3 out = currentPos;

//Rotate by x
//        (  1    0      0    )
//Rx(q) = (  0  cos q  sin q  )
//		  (  0 -sin q  cos q  )

out.y = (out.y * cos(alpha)) - out.z * sin(alpha);
out.z = (out.y * sin(alpha)) + out.z * cos(alpha);
out.x = out.x;

////Rotate by y
//        (cos q  0  -sin q )
//Ry(q) = (0      1    0    )
//        (sin q  0  cos q  )

//
out.z = (out.z * cos(beta)) - out.x * sin(beta);
out.x = (out.z * sin(beta)) + out.x * cos(beta);
out.y = out.y;

////Rotate by z

////          ( cos q  sin q  0 )
////(x,y,z) = (-sin q  cos q  0 )
////		  ( 0        0    1 )
//
out.x = (out.x * cos(gamma)) - out.y * sin(gamma);
out.y = (out.x * sin(gamma)) + out.y * cos(gamma);
out.z = out.z;

return out;
}


I'd really appreciate it if someone coudld let me know if they think they know where i've messed up.

##### Share on other sites
Well, in matrices your transfomation is T=R*P (T=transformation, R=rotation, P=position/translation matrix, X-1=inverse of X).
The inverted matrix is T-1 =(R*P)-1=P-1*R-1, as you see, you have to change the order of transformations.

So far your code looks good, but I believe that your rotation is messed up. The reason is, that you do euler rotations. Infact your rotation consists of 3 rotations R=R_alpha*R_beta*R_gamma. To invert this rotation you have to change the order too, in this case: R-1=(R_alpha*R_beta*R_gamma)-1=R_gamma-1*R_beta-1*R_alpha-1.

To solve your problem try to use an other RotateByOrigin function (i.e. RotateByOriginInv) which changes the order of sub rotations.

Good luck.

--
Ashaman

##### Share on other sites
So are you saying that rotating and "unrotating" are different? ie Rotate(x) + Rotate(-x) != 0?

I've started looking at finding the inverse of a rotation matrix, but my linear algebra is a bit rusty. Could someone help me go through the rest of calculating the inverse for one matrix?

say Rx(q) -> rotation about x-axis. Inverse matrix Rx(q)-1 is...

          [ (cos(q)^2) + (sin(q)^2)    0        0        ]         1Rx(q)-1 = [             0              cos(q)  -sin(q)   ]  *     ---          [            0               sin(q)  cos(q)    ]      |Rx(q)|

Firstly, is this correct? Also, how do i resolve the term in the top left of the equation? Can this be simplified?

And how do you get the magnitude of a matrix? Is it the root of the sum of the squares of the terms (as it is in a vector)?

EDIT: If (cos(q)^2) + (sin(q)^2) = 1, and you get the magnitude as I described...is Rx(q)-1 simply...

          [ 1         0        0     ]       Rx(q)-1 = [ 0      cos(q)  -sin(q)   ]            [ 0      sin(q)  cos(q)    ]

##### Share on other sites
Quote:
 Original post by WinegumsI've started looking at finding the inverse of a rotation matrix, but my linear algebra is a bit rusty. Could someone help me go through the rest of calculating the inverse for one matrix?
As long as you are either not using uniform scaling, or not using any scaling at all, then your 3x3 rotation matrix is orthogonal, and the inverse of an orthogonal matrix is the same as its transpose.

##### Share on other sites
Hi, I wrote a RotateByOriginInverse() function which inverts the rotation matricies and does the rotation calculations in the opposite order, but I'm still getting the same problem.

Interestingly the app seems to calculate anything along a slab parallel to the screens y-axis as a collision. Could I be missing something with the ray generation?

	Vector3 RotateByOrigin(const Vector3& currentPos, const float& alpha, const float& beta, const float& gamma)		{			Vector3 out = currentPos;			////Rotate by z			////          ( cos q  sin q  0 )			////(x,y,z) = (-sin q  cos q  0 )			////		  ( 0        0    1 )			//			out.x = (out.x * cos(gamma)) - out.y * sin(gamma);			out.y = (out.x * sin(gamma)) + out.y * cos(gamma);			out.z = out.z;				////Rotate by y			//        (cos q  0  -sin q )			//Ry(q) = (0      1    0    )			//        (sin q  0  cos q  )			//			out.z = (out.z * cos(beta)) - out.x * sin(beta);			out.x = (out.z * sin(beta)) + out.x * cos(beta);			out.y = out.y;			//Rotate by x			//        (  1    0      0    )			//Rx(q) = (  0  cos q  sin q  )			//		  (  0 -sin q  cos q  )			out.y = (out.y * cos(alpha)) - out.z * sin(alpha);			out.z = (out.y * sin(alpha)) + out.z * cos(alpha);			out.x = out.x;						return out;		}

##### Share on other sites
>So are you saying that rotating and "unrotating" are different? ie Rotate(x) + Rotate(-x) != 0?
No, for a single rotation Rotate(x)*Rotate(-x)=I(=0) is true for. But your rotation method is not one single, but 3 separated rotations. Working with euler-angles is always somewhat tricky ;)

In your last codefragment you didn't rename your function. This could be a problem if you use the same function to rotate your bounding boxed, in this case you annihalte the effect. Use two different functions, one for the rotation and one for the inverse rotation.

So far it looks good.

--
Ashaman

##### Share on other sites
Hi, sorry that is a typo, they are two different functions. However I'm not sure where I'm meant to be rotating and where I'm meant to be inverse rotating?

##### Share on other sites
Sorry for the bump, but I'm still quite lost here. Can anyone advise how I orientate the cube to be axis-aligned and rotate the ray appropriatley. I can't find anything on google...

##### Share on other sites
Quote:
 Original post by WinegumsSorry for the bump, but I'm still quite lost here. Can anyone advise how I orientate the cube to be axis-aligned and rotate the ray appropriatley. I can't find anything on google...

If you're using an AABB, don't rotate it, ever! [smile]

It's a bit hard to explain, but when you 'rotate' an Axis Aligned Bounding Box (AABB) it doesn't really rotate, but effectively adjusts it's bounds to fit the rotate shape. It remains axis aligned, giving you an increasingly poorer 'fit' to the original box the more you rotate it. Hopefully this 2D example will convey this clearer:

Your approach would indeed be correct to keep the AABBs aligned without rotations and to transform the ray into the AABB's local ('model') space. Unfortunately I don't know much about OpenGL's math utilities, but I'd imagine like DirectX it provides most matrix math functions to deal with this problem.

Basically your ray is in clip/view/camera space so you have to transform it back to 'world' space, in which the rotated AABBs would also theoretically reside, and from there back to the model space of the AABB. Transforming the ray back to world space is called unprojection, for which OpenGL seems to offer the glUnProject function.

Once the ray is unprojected into world space, you can transform it by the inverse of the AABBs world matrix (containing its rotation and translation). Then you can intersect this transformed ray with the plain AABB. What effectively happens is that AABB is put at the origin of its private local model space, without any rotations, and the ray is transformed to match. Perhaps a bit redundant, but like this:

Hope this helps :)

##### Share on other sites
@remigus: I think Winegums does the correct approach, transforming the ray into the object space of the AABB, but in a "hand" coded way.

@Wingums: My assumption is, that you have all your base data in world space. Then your transformation should work! Maybe there's an other bug. Whenever I encounter such bugs I always do some function test.

Start with simple tests, choose a simple line-start position (AABB.x,AABB.y+100) and set the line-end position to the center of the AABB. After transforming your line the line-end should be (0,0) and your line-start position (0,100). If not, what values do you encounter ? Is there some pattern (i.e. x always negated) ?

To answer your question where to use the standard rotation and where to use the inverse version, in this case you really just need to change your code(refering first post) from

//Rotate about originrotatedLine.mp0 = RotateByOrigin(rotatedLine.mp0,-alpha,-beta,-gamma);rotatedLine.mp1 = RotateByOrigin(rotatedLine.mp1,-alpha,-beta,-gamma);

to

//Rotate about originrotatedLine.mp0 = RotateByOriginInverse(rotatedLine.mp0,-alpha,-beta,-gamma);rotatedLine.mp1 = RotateByOriginInverse(rotatedLine.mp1,-alpha,-beta,-gamma);

Keep testing ;_)

--
Ashaman
--

1. 1
2. 2
3. 3
4. 4
frob
15
5. 5

• 14
• 12
• 20
• 12
• 13
• ### Forum Statistics

• Total Topics
632152
• Total Posts
3004470

×