hi gamedev forums, i'm not actually working on a game, yet looking for a solution to a problem most likely encountered in 3D game programming so i came here to ask for help. using OpenGL via Processing, there is a camera of which i know the location and the lookAt point. Also the up-vector is always 0,1,0. (y is probably reversed compared to standard OpenGL? doesn't really matter). This camera behaves mostly like a standard 3D shooter, it can move around, rotate horizontally and tilt up/down up to 90 degrees. no roll. now on the plane that is your screen panel, you have a mouse (or in this case, tablet) cursor. i need to be able to get the game world 3D coordinates of this mouse cursor in order to use it to draw things into the 3D space with the tablet pen. i'm having troubles to get the rotations right. so i've read a bit about Quaternions… and have concluded that i don't get them. is why i'm asking here now.

You can calculate the reverse transformation and multiple it by the mouse position, or just use gluUnProject to calculate it for you.

You don't need quaternions. What you're wanting to do is called 'picking', and is discussed fairly frequently here on the forums. A typical solution is to 'unproject' the mouse coordinates into world space, and use the resulting coordinate(s) to build a picking ray or segment. You can unproject the mouse coordinates at the near plane and use the camera position as the origin for the ray (which will work for perspective projections), or unproject at the near and far planes and construct a 'picking segment' (while will work for both orthographic and perspective projections).

For more info on the subject, try searching the forum archives for 'opengl picking'.

ah yes, i guess what i'm doing is only the first part of picking, as i don't even need to reach into space and calculate intersections. So if this 'unprojecting' takes window coordinates and unprojects them into 3d coordinates, it is exactly what i need.