Hey guys I'm trying to implement a simple ray picker in Opengl. In my little scene I have a green house that I would like to detect when the user clicks it. I am drawing the rays to the screen so I can debug them, so please have a look at these screenshots:
The red lines are the rays I have cast. The places I clicked are in the first screenshot on the outer endpoints of each ray. The problem is this projection to the far plane does not give accurate results if my goal is to determine if the user clicked the house or not. Should I be orthogonally projecting the rays from their origins? What can I do to fix this? Here is my code:
Ray computePickingRay(int x, int y)
{
glm::mat4 viewProj = projectionMatrix * camera->getViewMatrix();
glm::mat4 invViewProj = glm::inverse(viewProj);
int sx = gScreenWidth / 2;
int sy = gScreenHeight / 2;
glm::vec4 inV;
inV.x = 1.0f * (x - sx) / (1.0f*sx);
inV.y = 1.0f * ((gScreenHeight - y) - sy) / (1.0f * sy);
inV.z = -1.0f;
inV.w = 1.0f;
glm::vec4 nearPlane = invViewProj * inV;
inV.z = 1.0f;
glm::vec4 farPlane = invViewProj * inV;
glm::vec4 dir = (farPlane - nearPlane);
Ray ray;
ray.origin = vec3(nearPlane.x/nearPlane.w, nearPlane.y/nearPlane.w, nearPlane.z/nearPlane.w);
ray.direction = glm::normalize(-vec3(dir.x/dir.w, dir.y/dir.w,dir.z/dir.w));
return ray;
}
