Sign in to follow this  
donlucacorleone

[SOLVED] Move 3D object with 2D mouse

Recommended Posts

Ok, I know it's a common question. I want to move a DirectX object with the mouse cursor. But I'm missing something (maybe basic 3D math). On mouse click I want to move an object, previously rendered somewhere, to a NewX, NewY, SameZ values. The Z is the same of the actual Z of the object, so I don't need to calculate it. But I need to compute the NewX and NewY values... This is what I've done ( I use this for picking, see: http://www.gamedev.net/community/forums/topic.asp?topic_id=547663).
D3DXVECTOR3 get3dWorldCoordFrom2dPoint(float point2dX, float point2dY, float objectZCoord, D3DXMATRIX worldMatrix)
{
	//Get the Viewport
	D3D10_VIEWPORT viewport;
	UINT numOfVP = 1;
	G_Direct3D->device->RSGetViewports(&numOfVP, &viewport);

	//First point of a ray
	D3DXVECTOR3 point3d;
	point3d.x = (float) point2dX;
	point3d.y = (float) point2dY;
	point3d.z = -100.0f;

	//Translate from 2d to 3d World
	D3DXVECTOR3 rayStart;
	D3DXVec3Unproject(&rayStart, &point3d, &viewport, &G_Direct3D->projectionMatrix, &G_Direct3D->viewMatrix, &worldMatrix);

	//Second point of a ray
	point3d.z = 100.0f;

	//Translate from 2d to 3d World
	D3DXVECTOR3 rayEnd;
	D3DXVec3Unproject(&rayEnd, &point3d, &viewport, &G_Direct3D->projectionMatrix, &G_Direct3D->viewMatrix, &worldMatrix);

// --- And now ???
//I've copied this code from the web, but I really don't understand it
	float point3dX = ( ( 0.0f - rayStart.z ) * ( rayEnd.x - rayStart.x ) ) / ( rayEnd.z - rayStart.z ) + rayStart.x;
	float point3dY = ( ( 0.0f - rayStart.z ) * ( rayEnd.y - rayStart.y ) ) / ( rayEnd.z - rayStart.z ) + rayStart.y;
	float point3dZ = objectZCoord;


	return D3DXVECTOR3 (point3dX, point3dY, point3dZ);
}
As you can see, I don't exactly know how to get the 3D point from the ray. Perhaps I've to get the intersection between the ray and the plane, but I don't know the equation of the plane... I'm a bit confused, especially with the math beyond all that stuff. Could you please help me? Thank you. [Edited by - donlucacorleone on September 28, 2009 8:18:40 AM]

Share this post


Link to post
Share on other sites
Let's see if I understand this right: You want to click on a point on a level plane with height SameZ and get the 3D coordinates of this point?

If so, the two lines

float point3dX = ( ( 0.0f - rayStart.z ) * ( rayEnd.x - rayStart.x ) ) / ( rayEnd.z - rayStart.z ) + rayStart.x;
float point3dY = ( ( 0.0f - rayStart.z ) * ( rayEnd.y - rayStart.y ) ) / ( rayEnd.z - rayStart.z ) + rayStart.y;


are almost right. You see, the normal of your plane is n=(0,0,1), its distance d to the origin is "objectZCoord". The formula you used can be looked up at Line-plane intersection (section: "Algebraic form"). Just replace the 0.0f by -objectZCoord and you should be good to go (I think).

Share this post


Link to post
Share on other sites
Thank you Nimbal for your response.

Let me explain better:
When I click the mouse, I want to re-render an object in that position ("under" the mouse cursor).
The new Z of the object is the same that was previously setted.
I need to get the NewX and NewY, in World coordinates obviously.

Making your changes, it doesn't work. It works only when the SameZ is at 0.0f :-(

I've tried a lot of plane-line intersection forumulas, but I can't catch the right way to implement it in my scenario.


--- EDIT ---
After reading this topic: http://www.gamedev.net/community/forums/topic.asp?topic_id=321357&whichpage=1�

I rewrite my function with this piece of code:

float t = - ( D3DXVec3Dot(&n, &rayStart) + d ) / ( D3DXVec3Dot(&n, &normalizedRayDir) );
D3DXVECTOR3 new3dPoint = rayStart + t * normalizedRayDir;


where "n" is the normal (that is (0,0,1)) and "d" is equal to "objectZCoord".

Again, it works only if the objectZCoord is 0 (zero)...

[Edited by - donlucacorleone on September 22, 2009 8:04:02 AM]

Share this post


Link to post
Share on other sites
When Z is not zero, the object is rendered near the mouse icon, but not "under" it.
Especially: if the mouse is at the top left corner of the window, then the render of the object starts at a negative offset on the X and at a positive offset on the Y.
In the bottom right corner, these offsets are reversed...

Maybe it's due to the FOV of the Projection matrix ??

Share this post


Link to post
Share on other sites
Are you sure that setting point3d.z (the depth of the mouse cursor position) to plus/minus 100 is right? As far as I know, 0.0 and 1.0 are the values for near and far plane respectively. Try those and see if anything changes.

Share this post


Link to post
Share on other sites
You're right! Thank you Nimbal.

I was using these values because the picking function doesn't work with 0.0f and 1.0f.
Now I'll correct and analyze both functions.

Thank you again, you solved it.

Ah, just to be as clear as possible, in this piece of code

float point3dX = ( ( 0.0f - rayStart.z ) * ( rayEnd.x - rayStart.x ) ) / ( rayEnd.z - rayStart.z ) + rayStart.x;
float point3dY = ( ( 0.0f - rayStart.z ) * ( rayEnd.y - rayStart.y ) ) / ( rayEnd.z - rayStart.z ) + rayStart.y;



the 0.0f means the origin. Am i right?

--- EDIT ---
Another information: you should substitute the 0.0f and 1.0f values with viewport.MinDepth and viewport.MaxDepth respectively

Share this post


Link to post
Share on other sites
Thank you for the link Jose.

So, my new test code is:

//Plane with Z = objectZCoord
float A = 0.0f;
float B = 0.0f;
float C = -1.0f;
float D = objectZCoord;
D3DXVECTOR3 R0 = rayStart;
D3DXVECTOR3 Rd = rayEnd;

//Already tested and they're ok
D3DXVECTOR3 Pn = D3DXVECTOR3(A, B, C);
float Vd = D3DXVec3Dot(&Pn, &Rd);
float V0 = - ( D3DXVec3Dot(&Pn, &R0) + D);
float t = V0 / Vd;

float Xi = R0.x + Rd.x * t;
float Yi = R0.y + Rd.y * t;
float Zi = R0.z + Rd.z * t;




rayStart and rayEnd were calculated using D3DXVec3Unproject() already posted previously. I *think* (but now I've no more certainties) that they're ok.

The Zi value is equal (except for a little approximation) to the objectZCoord.
But the Xi and Yi values aren't correct. For "correct" I mean that they are not under my mouse icon on screen...

Am I missing something??


If you want I can post some real values to play with.

Share this post


Link to post
Share on other sites
If I understand you correctly, you have a plane described by normal=(0,0,-1) and a z intercept of objectZCoord. You want to obtain the world coordinates X & Y on that plane under the mouse pointer. I think this should work.

POINT mousePt;
D3DXVECTOR3 v0, v1;
D3DXVECTOR3 rayPos, rayDir;

v0 = D3DXVECTOR3(mousePt.x, mousePt.y, 0);
v1 = D3DXVECTOR3(mousePt.x, mousePt.y, 1);
D3DXVec3Unproject(&rayPos, &v0, &vp, &proj, &view, &world);
D3DXVec3Unproject(&rayDir, &v1, &vp, &proj, &view, &world);
rayDir -= rayPos;
D3DXVec3Normalize(&rayDir,&rayDir);
// rayDir is a vector in 3D world from eye point to infinity
// define D3DXVECTOR3 wPos = rayPos+factor*rayDir with wPos a point in the plane
// you want to solve for factor
// wPos.z = rayPos.z + factor*rayDir.z
// wPos.z = objectZCoord // since it's in the plane
// CHECK FOR RAYDIR.Z==0 to avoid division by zero
// This happens when you're looking parallel to the plane
// factor = (wPos.z - rayPos.z)/rayDir.z
// wPos.x = rayPos.x + rayDir.x*factor // desired x value
// wPos.y = rayPos.y + rayDir.y*factor // desired y value

EDIT: check for negative "factor" values for the case where the ray direction is pointing away from the plane rather than toward it.


[Edited by - Buckeye on September 23, 2009 7:15:38 AM]

Share this post


Link to post
Share on other sites
You've understood my problem.
But your method doesn't work.

I post a numerical example.

1) Put an object (formed by two triangles) in the world described by:

//World Matrix:
1 0 -0 0
0 1 0 0
0 0 1 0
0 0 410 1
//410 is the objectZCoord

2) Mouse click happens in 784, 583
3) New world coordinates returned by my function: 747, -550, 410
4) Your function returns 964, -711, 410

On screen, my function renders the object exactly under the mouse icon, yours doesn't. There's always an offset (except for objectZCoord = 0)

I don't know if it helps you to understand better, I hope so.

I'm a bit confusing and I need your help...

Share this post


Link to post
Share on other sites
Quote:
Original post by Buckeye
My code assumes the world matrix is the device matrix obtained with dev->GetTransform(D3DTS_WORLD,&world), not the model matrix.


Uh?! This is what MSDN says and this is my religion too:
World transformation, as the name suggests, converts vertices from object space to world space. It usually consists of one or more scaling, rotation, and translation, based on the size, orientation, and position we would like to give to the object. Every object in the scene has its own world transformation matrix. This is because each object has its own size, orientation, and position.

Also I'm using DirectX 10 that hasn't got the Device->GetTransform method.

By the way, assuming that matrix (the one I've posted) is your "device" matrix, your code should work without problem.
In my case I've one object that I'd like to put in (Some_X_Value, Some_Y_Value, 410). So I need to calculate the Some_X_Value and Some_Y_Value which represent the mouse icon projected in 3D world.

I hope I've explained it more clearly. Sorry if it isn't so clear but I'm not english.

Thank you.

Share this post


Link to post
Share on other sites
Quote:
I'm using DirectX 10 that hasn't got the Device->GetTransform method.
Okay. Use an identity matrix for the calc.
Quote:
I need to calculate the Some_X_Value and Some_Y_Value which represent the mouse icon projected in 3D world.

Your original calc uses the object's transform to calculate a ray position and direction in the object's space, not the 3D world space. Use an identity matrix for that first unproject calc to get the vector in world space. Then use objectZCoord (the object's transform) to transform the ray vector. In your original calc, you've applied the object's transform twice.

Your original calc is calculating the X and Y position of the mouse as if it were moved to the object's position, then moving it again by the same amount. The reason your calculated positions weren't precise was because it calculated the X and Y positions as if the object was at 410 + 410, not just 410.

Share this post


Link to post
Share on other sites
Quote:

Your original calc uses the object's transform to calculate a ray position and direction in the object's space, not the 3D world space

I disagree with you, maybe beacuse I'm really getting cunfused...

I repost my simplified code. This is the function which purpose is Get a 3D ray (that is two 3D points) from a 2D point:


D3DXVECTOR3 temp3dPoint;
temp3dPoint.x = (float) point2dX;
temp3dPoint.y = (float) point2dY;
temp3dPoint.z = viewport.MaxDepth;

D3DXVECTOR3 rayStart;
D3DXVec3Unproject(&rayStart, &temp3dPoint, &viewport, &G_Direct3D->projectionMatrix, &G_Direct3D->viewMatrix, &worldMatrix);


temp3dPoint.z = viewport.MinDepth;

D3DXVECTOR3 rayEnd;
D3DXVec3Unproject(&rayEnd, &temp3dPoint, &viewport, &G_Direct3D->projectionMatrix, &G_Direct3D->viewMatrix, &worldMatrix);



What is "worldMatrix" for you? I don't think it's always an identity matrix. Perhaps it'd be if there weren't translations, rotation, ecc.
Help me please...

Share this post


Link to post
Share on other sites
One more try. I'm trying to help you, but if you don't agree, that's up to you.

1. What are you using for the worldMatrix? If you're using the object's matrix, you're positioning the vector in your object's space, not the world space.

2. The ray should start at the near (minimum value) not the far (maximum) value.

3. The ray should end at the far value, not the near value.

Share this post


Link to post
Share on other sites
Hey Buckeye, I was trying to say that I disagree with you only beacuse I don't understand what you wrote. It wasn't my intention to say that you're wrong. Apologize me for this misunderstanding.

I add another information.
You (Ripiz and Buckeye) have already helped me in "Picking 3D mesh with mouse" (http://www.gamedev.net/community/forums/topic.asp?topic_id=547663).

In that case the "worldMatrix" was the object world matrix, right?! I hope so... Ok, call that object "A".

Now, in this case, I need to put "A" in another 3D world, for example the world of object "B". So my answers are:

1 - "B" object world matrix.
2 & 3 - Of course! They were typing errors.

Thank you again for your help, I've really appreciated it.

Share this post


Link to post
Share on other sites
Quote:
In that case the "worldMatrix" was the object world matrix, right?!

Apparently. The matrixWorld you used in that post worked correctly because it was really world(Identity)Matrix*objectMatrix.

In any case, that isn't relevant to your question in this post, which was to locate the x/y values in the 3D world for a known z value. How you obtained that z-value doesn't make any difference to that calc, and you shouldn't be using the object's matrix at all for that calc. It's a simple matter of determining a point in 3D space.

If you had originally asked, "How do I find the x/y coordinates under the mouse for any z-value?," you'd have gotten the same answer.

Once the x/y values are calculated, then you can move your object.

Share this post


Link to post
Share on other sites
Quote:

It's a simple matter of determining a point in 3D space

Yes! I'm asking how can I find the (x,y) 3D world coordinates from a given 2D mouse point and a given 3D Z value...

[input]
2D mouse point
Z value
[output]
The 3D World coordinates in which I've to put my object, so the user sees it under the mouse

The code you've posted is right (only now I understand it).

POINT mousePt;
D3DXVECTOR3 v0, v1;
D3DXVECTOR3 rayPos, rayDir;
v0 = D3DXVECTOR3(mousePt.x, mousePt.y, 0);
v1 = D3DXVECTOR3(mousePt.x, mousePt.y, 1);
D3DXVec3Unproject(&rayPos, &v0, &vp, &proj, &view, &world);
D3DXVec3Unproject(&rayDir, &v1, &vp, &proj, &view, &world);
rayDir -= rayPos;
D3DXVec3Normalize(&rayDir,&rayDir);
// rayDir is a vector in 3D world from eye point to infinity
// define D3DXVECTOR3 wPos = rayPos+factor*rayDir with wPos a point in the plane
// you want to solve for factor
// wPos.z = rayPos.z + factor*rayDir.z
// wPos.z = objectZCoord // since it's in the plane
// CHECK FOR RAYDIR.Z==0 to avoid division by zero
// This happens when you're looking parallel to the plane
// factor = (wPos.z - rayPos.z)/rayDir.z
// wPos.x = rayPos.x + rayDir.x*factor // desired x value
// wPos.y = rayPos.y + rayDir.y*factor // desired y value



The issue is related to that damned "worldMatrix" :-).
Really, I don't understand why I've to put an Identity matrix instead of the matrix that describes the world in which I'd put the object.

Please be patient with me, I'm only an unexpert DirectX developer and all these stuff are getting me crazy.

Again, thank you for your help.

Share this post


Link to post
Share on other sites
First, your object has nothing to do with the calculation of the point (except that's where you got the z-value).

The identity matrix is the world matrix into which you put your object. The object or model matrix moves the object within that world. Don't think of the model matrix as a "world" matrix. It describes where the model's local space is located in the world.

One way to think of it (until you understand the math better):

When you created your object, you created it at (0,0,0), at the origin of the world. If you used an identity model matrix, that's where it would be located. But you didn't want it located at the origin, so you specified a location in the world described by the model matrix.

To pick an object, you create a ray in the world space. Then you determine if it intersects an object located in the world at a particular location (described by the model matrix).

Now that you've located a point on the object in the world, you want to find another point in the world near that location in the world because the mouse moved. At this point, it doesn't matter how you found the location. So you do the calculation of the new point in the world near the old one.

You have a new location and you move the model in the world based on that result.

[Edited by - Buckeye on September 25, 2009 10:35:38 AM]

Share this post


Link to post
Share on other sites
Quote:
Original post by Buckeye
First, your object has nothing to do with the calculation of the point (except that's where you got the z-value).

The identity matrix is the world matrix into which you put your object. The object or model matrix moves the object within that world. Don't think of the model matrix as a "world" matrix. It describes where the model's local space is located in the world.

One way to think of it (until you understand the math better):

When you created your object, you created it at (0,0,0), at the origin of the world. If you used an identity model matrix, that's where it would be located. But you didn't want it located at the origin, so you specified a location in the world described by the model matrix.

To pick an object, you create a ray in the world space. Then you determine if it intersects an object located in the world at a particular location (described by the model matrix).

Now that you've located a point on the object in the world, you want to find another point in the world near that location in the world because the mouse moved. At this point, it doesn't matter how you found the location. So you do the calculation of the new point in the world near the old one.

You have a new location and you move the model in the world based on that result.


Now I really understand!
And now my routine works as expected!

Thank you very much Buckeye!

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this