I'm trying to build a small game in Unity and I ran into this problem. I want to use an unlocked cursor in the game meaning that the look at vector of the camera would not exactly point to the cursor. This is the default setting in Unity it's done by not locking the cursor the center of the screen and simply using the offset, instead the camera moves by the distance that the user moved the cursor. This allows me to use a hardware cursor in the game and also I kind of like the feel of it since the game I have in mind involves a lot of 3D picking. So the problem is that when I try to actually shoot a ray from camera through the cursor (converted to 3D position) the shooting vector is off by a lot at certain camera angles and very close to what it's supposed to be at other angles. I'm not sure what's causing it, this is the code I have so far (except from a Unity C# script)

vector3 v = Camera.ScreenPointToRay(mousePos)
ray = v.direction;


I'm not exactly sure how to fix this or even why it's happening. I suspect that the conversion of the 2D cursor to 3D space results in a false Z value in some camera orientation but I don't exactly how to fix it. Any help appreciated

From the documentation, ScreenPointToRay takes a screen position in pixels. Are-you sure that your mousePos is in pixels? It's not clear from your post what the 2D to 3D converstion your talking about exactly is. As far as I know, you shouldn't have any 2D to 3D conversion to do.

I meant converting a 2D cursor postion to 3D world space, at least that's what I thought I had to do. I just noticed the doc says that z coordinate is ignored, but then how does it build a ray if the z is ignored? And yes the mousepos is in pixels just checked the doc Input.mousePosition. I hope that I stated my original problem clearly. I basically want to use the actual distance that the mouse moved instead of the deltaX deltaY to control the camera view. So you could still move the cursor normally and it would feel like a normal cursor rather then crosshairs in an fps.

Well, a ray is defined by two things: its start position and a direction. The start position is the camera position and the direction is what is returned by ScreenPointToRay. The z component of the mouse position is useless, because the direction will be the same for any point in space that appears to be at that pixel position. I don't know if I'm clear enough. Think of it as a wire starting from the camera and going away to infinity. Any point on this wire has the same pixel position but different z values. That means that we can ignore the z value to get the ray direction.

You made an example of a wire going from the camera to infinity, and I'm assuming the camera's lookAt vector 'z' is used to make the 3D ray that points in that direction but also has all the 3D points that that pixel contains because as you said it remains constant. But isn't that assuming that the camera lookAt vector is pointing at the object on which you clicked? Please correct if I'm wrong because I very well could be. But assuming that's so, in this camera model I'm trying my pointer doesn't point to the same place the character is looking at because by the time I drag the cursor from left to right of the screen, the game character would have made a few turns around the up axis. This way the cursor ends up in random places like being at the very bottom of the screen when the character is looking directly up or could be in upper left of the screen when he's facing directly forward. Assuming all of this how then can I get the actual direction point I'm clicking on? Again correct me if I said anything off, I'm just starting trying to understand raycasting and picking as I'm going through this book.

anyone?

The look at direction of the camera is the ray direction for the center of the screen only, for the other pixels, you need to call ScreenPointToRay. I'm really not sure I understand what you mean, so please try to explain what you're trying to do with more details or post some code.

By the way, your initial example is wrong, the ScreenPointToRay method returns a Ray, not a vector. Did you really use this code?

I've been using ScreeinPointToRay and it doesn't work. The ray that I get doesn't go from camera to the object I hit. I don't know how else to explain it. Imagine a character facing directly forward, in most games that would also mean that the cursor or crosshairs are also directly in the middle of the screen. I could have my cursor be located at ANY point on the screen while the character is still facing forward. So lets say I click the mouse button while pointing at a rock on the ground (rember the character is facing forward so the rock is at the lower right corner of the screen). How would I get a ray that goes from camera to rock?

I guess I should have posted code earlier but I wanted to see if I could understand what's going on first, it would be pretty embarassing if I've been using this simple function in the wrong way all this time... Allright so here's the main code, it's attached to my Player character:

[source lang="csharp"]
//Activates when user presses a left mouse button
if(Input.GetButtonUp ("Fire1"))
{
Rigidbody bulletInst = Instantiate(bullet, transform.position, transform.rotation) as Rigidbody;
Ray camToCursorRay = Camera.main.ScreenPointToRay(Input.mousePosition);
shotDirection = camToCursorRay.direction; //public global float used to monitor this var
bulletInst.AddForce(shotDirection * shotPower); //shotPower multiplier set to 1500f


}[/source]