• Create Account

## [SOLVED] Move 3D object with 2D mouse

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

22 replies to this topic

### #1donlucacorleone  Members

Posted 21 September 2009 - 10:55 PM

Ok, I know it's a common question. I want to move a DirectX object with the mouse cursor. But I'm missing something (maybe basic 3D math). On mouse click I want to move an object, previously rendered somewhere, to a NewX, NewY, SameZ values. The Z is the same of the actual Z of the object, so I don't need to calculate it. But I need to compute the NewX and NewY values... This is what I've done ( I use this for picking, see: http://www.gamedev.net/community/forums/topic.asp?topic_id=547663).
D3DXVECTOR3 get3dWorldCoordFrom2dPoint(float point2dX, float point2dY, float objectZCoord, D3DXMATRIX worldMatrix)
{
//Get the Viewport
D3D10_VIEWPORT viewport;
UINT numOfVP = 1;
G_Direct3D->device->RSGetViewports(&numOfVP, &viewport);

//First point of a ray
D3DXVECTOR3 point3d;
point3d.x = (float) point2dX;
point3d.y = (float) point2dY;
point3d.z = -100.0f;

//Translate from 2d to 3d World
D3DXVECTOR3 rayStart;
D3DXVec3Unproject(&rayStart, &point3d, &viewport, &G_Direct3D->projectionMatrix, &G_Direct3D->viewMatrix, &worldMatrix);

//Second point of a ray
point3d.z = 100.0f;

//Translate from 2d to 3d World
D3DXVECTOR3 rayEnd;
D3DXVec3Unproject(&rayEnd, &point3d, &viewport, &G_Direct3D->projectionMatrix, &G_Direct3D->viewMatrix, &worldMatrix);

// --- And now ???
//I've copied this code from the web, but I really don't understand it
float point3dX = ( ( 0.0f - rayStart.z ) * ( rayEnd.x - rayStart.x ) ) / ( rayEnd.z - rayStart.z ) + rayStart.x;
float point3dY = ( ( 0.0f - rayStart.z ) * ( rayEnd.y - rayStart.y ) ) / ( rayEnd.z - rayStart.z ) + rayStart.y;
float point3dZ = objectZCoord;

return D3DXVECTOR3 (point3dX, point3dY, point3dZ);
}

As you can see, I don't exactly know how to get the 3D point from the ray. Perhaps I've to get the intersection between the ray and the plane, but I don't know the equation of the plane... I'm a bit confused, especially with the math beyond all that stuff. Could you please help me? Thank you. [Edited by - donlucacorleone on September 28, 2009 8:18:40 AM]

### #2Nimbal  Members

Posted 22 September 2009 - 12:41 AM

Let's see if I understand this right: You want to click on a point on a level plane with height SameZ and get the 3D coordinates of this point?

If so, the two lines
float point3dX = ( ( 0.0f - rayStart.z ) * ( rayEnd.x - rayStart.x ) ) / ( rayEnd.z - rayStart.z ) + rayStart.x;float point3dY = ( ( 0.0f - rayStart.z ) * ( rayEnd.y - rayStart.y ) ) / ( rayEnd.z - rayStart.z ) + rayStart.y;

are almost right. You see, the normal of your plane is n=(0,0,1), its distance d to the origin is "objectZCoord". The formula you used can be looked up at Line-plane intersection (section: "Algebraic form"). Just replace the 0.0f by -objectZCoord and you should be good to go (I think).

### #3donlucacorleone  Members

Posted 22 September 2009 - 02:04 AM

Thank you Nimbal for your response.

Let me explain better:
When I click the mouse, I want to re-render an object in that position ("under" the mouse cursor).
The new Z of the object is the same that was previously setted.
I need to get the NewX and NewY, in World coordinates obviously.

Making your changes, it doesn't work. It works only when the SameZ is at 0.0f :-(

I've tried a lot of plane-line intersection forumulas, but I can't catch the right way to implement it in my scenario.

--- EDIT ---

I rewrite my function with this piece of code:
float t = - ( D3DXVec3Dot(&n, &rayStart) + d ) / ( D3DXVec3Dot(&n, &normalizedRayDir) );D3DXVECTOR3 new3dPoint = rayStart + t * normalizedRayDir;

where "n" is the normal (that is (0,0,1)) and "d" is equal to "objectZCoord".

Again, it works only if the objectZCoord is 0 (zero)...

[Edited by - donlucacorleone on September 22, 2009 8:04:02 AM]

### #4Nimbal  Members

Posted 22 September 2009 - 02:47 AM

What happens when d is not zero (or rather near zero, but not quite)? In which direction is the object misplaced?

### #5donlucacorleone  Members

Posted 22 September 2009 - 02:55 AM

When Z is not zero, the object is rendered near the mouse icon, but not "under" it.
Especially: if the mouse is at the top left corner of the window, then the render of the object starts at a negative offset on the X and at a positive offset on the Y.
In the bottom right corner, these offsets are reversed...

Maybe it's due to the FOV of the Projection matrix ??

### #6Nimbal  Members

Posted 22 September 2009 - 06:02 AM

Are you sure that setting point3d.z (the depth of the mouse cursor position) to plus/minus 100 is right? As far as I know, 0.0 and 1.0 are the values for near and far plane respectively. Try those and see if anything changes.

### #7donlucacorleone  Members

Posted 22 September 2009 - 08:28 PM

You're right! Thank you Nimbal.

I was using these values because the picking function doesn't work with 0.0f and 1.0f.
Now I'll correct and analyze both functions.

Thank you again, you solved it.

Ah, just to be as clear as possible, in this piece of code
float point3dX = ( ( 0.0f - rayStart.z ) * ( rayEnd.x - rayStart.x ) ) / ( rayEnd.z - rayStart.z ) + rayStart.x;float point3dY = ( ( 0.0f - rayStart.z ) * ( rayEnd.y - rayStart.y ) ) / ( rayEnd.z - rayStart.z ) + rayStart.y;

the 0.0f means the origin. Am i right?

--- EDIT ---
Another information: you should substitute the 0.0f and 1.0f values with viewport.MinDepth and viewport.MaxDepth respectively

### #8Nimbal  Members

Posted 22 September 2009 - 08:45 PM

It should be the distance between the plane and the origin (so, in your case objectZCoord).

### #9donlucacorleone  Members

Posted 22 September 2009 - 09:05 PM

Are you sure?
Removing 0.0f and setting objectZCoord it'll not work anymore except when objectZCoord = 0.0f ...

### #10joserodolfo  Members

Posted 22 September 2009 - 09:12 PM

http://www.siggraph.org/education/materials/HyperGraph/raytrace/rayplane_intersection.htm

### #11donlucacorleone  Members

Posted 22 September 2009 - 09:58 PM

Thank you for the link Jose.

So, my new test code is:
//Plane with Z = objectZCoordfloat A = 0.0f;float B = 0.0f;float C = -1.0f;float D = objectZCoord;D3DXVECTOR3 R0 = rayStart;D3DXVECTOR3 Rd = rayEnd;//Already tested and they're okD3DXVECTOR3 Pn = D3DXVECTOR3(A, B, C);float Vd = D3DXVec3Dot(&Pn, &Rd);float V0 = - ( D3DXVec3Dot(&Pn, &R0) + D);float t = V0 / Vd;float Xi = R0.x + Rd.x * t;float Yi = R0.y + Rd.y * t;float Zi = R0.z + Rd.z * t;

rayStart and rayEnd were calculated using D3DXVec3Unproject() already posted previously. I *think* (but now I've no more certainties) that they're ok.

The Zi value is equal (except for a little approximation) to the objectZCoord.
But the Xi and Yi values aren't correct. For "correct" I mean that they are not under my mouse icon on screen...

Am I missing something??

If you want I can post some real values to play with.

### #12Buckeye  GDNet+

Posted 23 September 2009 - 12:15 AM

If I understand you correctly, you have a plane described by normal=(0,0,-1) and a z intercept of objectZCoord. You want to obtain the world coordinates X & Y on that plane under the mouse pointer. I think this should work.
POINT mousePt;D3DXVECTOR3 v0, v1;D3DXVECTOR3 rayPos, rayDir;v0 = D3DXVECTOR3(mousePt.x, mousePt.y, 0);v1 = D3DXVECTOR3(mousePt.x, mousePt.y, 1);D3DXVec3Unproject(&rayPos, &v0, &vp, &proj, &view, &world);D3DXVec3Unproject(&rayDir, &v1, &vp, &proj, &view, &world);rayDir -= rayPos;D3DXVec3Normalize(&rayDir,&rayDir);// rayDir is a vector in 3D world from eye point to infinity// define D3DXVECTOR3 wPos = rayPos+factor*rayDir with wPos a point in the plane// you want to solve for factor// wPos.z = rayPos.z + factor*rayDir.z// wPos.z = objectZCoord // since it's in the plane// CHECK FOR RAYDIR.Z==0 to avoid division by zero// This happens when you're looking parallel to the plane// factor = (wPos.z - rayPos.z)/rayDir.z// wPos.x = rayPos.x + rayDir.x*factor // desired x value// wPos.y = rayPos.y + rayDir.y*factor // desired y value

EDIT: check for negative "factor" values for the case where the ray direction is pointing away from the plane rather than toward it.

[Edited by - Buckeye on September 23, 2009 7:15:38 AM]

### #13donlucacorleone  Members

Posted 23 September 2009 - 02:01 AM

You've understood my problem.

I post a numerical example.

1) Put an object (formed by two triangles) in the world described by:
//World Matrix:1 0 -0  00 1 0   00 0 1   00 0 410 1//410 is the objectZCoord

2) Mouse click happens in 784, 583
3) New world coordinates returned by my function: 747, -550, 410
4) Your function returns 964, -711, 410

On screen, my function renders the object exactly under the mouse icon, yours doesn't. There's always an offset (except for objectZCoord = 0)

I don't know if it helps you to understand better, I hope so.

I'm a bit confusing and I need your help...

### #14Buckeye  GDNet+

Posted 23 September 2009 - 06:43 AM

My code assumes the world matrix is the device matrix obtained with dev->GetTransform(D3DTS_WORLD,&world), not the model matrix.

### #15donlucacorleone  Members

Posted 23 September 2009 - 08:31 PM

Quote:
 Original post by BuckeyeMy code assumes the world matrix is the device matrix obtained with dev->GetTransform(D3DTS_WORLD,&world), not the model matrix.

Uh?! This is what MSDN says and this is my religion too:
World transformation, as the name suggests, converts vertices from object space to world space. It usually consists of one or more scaling, rotation, and translation, based on the size, orientation, and position we would like to give to the object. Every object in the scene has its own world transformation matrix. This is because each object has its own size, orientation, and position.

Also I'm using DirectX 10 that hasn't got the Device->GetTransform method.

By the way, assuming that matrix (the one I've posted) is your "device" matrix, your code should work without problem.
In my case I've one object that I'd like to put in (Some_X_Value, Some_Y_Value, 410). So I need to calculate the Some_X_Value and Some_Y_Value which represent the mouse icon projected in 3D world.

I hope I've explained it more clearly. Sorry if it isn't so clear but I'm not english.

Thank you.

### #16Buckeye  GDNet+

Posted 24 September 2009 - 01:55 AM

Quote:
 I'm using DirectX 10 that hasn't got the Device->GetTransform method.
Okay. Use an identity matrix for the calc.
Quote:
 I need to calculate the Some_X_Value and Some_Y_Value which represent the mouse icon projected in 3D world.

Your original calc uses the object's transform to calculate a ray position and direction in the object's space, not the 3D world space. Use an identity matrix for that first unproject calc to get the vector in world space. Then use objectZCoord (the object's transform) to transform the ray vector. In your original calc, you've applied the object's transform twice.

Your original calc is calculating the X and Y position of the mouse as if it were moved to the object's position, then moving it again by the same amount. The reason your calculated positions weren't precise was because it calculated the X and Y positions as if the object was at 410 + 410, not just 410.

### #17donlucacorleone  Members

Posted 24 September 2009 - 03:14 AM

Quote:
 Your original calc uses the object's transform to calculate a ray position and direction in the object's space, not the 3D world space

I disagree with you, maybe beacuse I'm really getting cunfused...

I repost my simplified code. This is the function which purpose is Get a 3D ray (that is two 3D points) from a 2D point:

D3DXVECTOR3 temp3dPoint;temp3dPoint.x = (float) point2dX;temp3dPoint.y = (float) point2dY;temp3dPoint.z = viewport.MaxDepth;D3DXVECTOR3 rayStart;D3DXVec3Unproject(&rayStart, &temp3dPoint, &viewport, &G_Direct3D->projectionMatrix, &G_Direct3D->viewMatrix, &worldMatrix);temp3dPoint.z = viewport.MinDepth;D3DXVECTOR3 rayEnd;D3DXVec3Unproject(&rayEnd, &temp3dPoint, &viewport, &G_Direct3D->projectionMatrix, &G_Direct3D->viewMatrix, &worldMatrix);

What is "worldMatrix" for you? I don't think it's always an identity matrix. Perhaps it'd be if there weren't translations, rotation, ecc.

### #18Buckeye  GDNet+

Posted 24 September 2009 - 04:18 AM

One more try. I'm trying to help you, but if you don't agree, that's up to you.

1. What are you using for the worldMatrix? If you're using the object's matrix, you're positioning the vector in your object's space, not the world space.

2. The ray should start at the near (minimum value) not the far (maximum) value.

3. The ray should end at the far value, not the near value.

### #19donlucacorleone  Members

Posted 24 September 2009 - 08:35 PM

Hey Buckeye, I was trying to say that I disagree with you only beacuse I don't understand what you wrote. It wasn't my intention to say that you're wrong. Apologize me for this misunderstanding.

You (Ripiz and Buckeye) have already helped me in "Picking 3D mesh with mouse" (http://www.gamedev.net/community/forums/topic.asp?topic_id=547663).

In that case the "worldMatrix" was the object world matrix, right?! I hope so... Ok, call that object "A".

Now, in this case, I need to put "A" in another 3D world, for example the world of object "B". So my answers are:

1 - "B" object world matrix.
2 & 3 - Of course! They were typing errors.

Thank you again for your help, I've really appreciated it.

### #20Buckeye  GDNet+

Posted 25 September 2009 - 03:05 AM

Quote:
 In that case the "worldMatrix" was the object world matrix, right?!

Apparently. The matrixWorld you used in that post worked correctly because it was really world(Identity)Matrix*objectMatrix.

In any case, that isn't relevant to your question in this post, which was to locate the x/y values in the 3D world for a known z value. How you obtained that z-value doesn't make any difference to that calc, and you shouldn't be using the object's matrix at all for that calc. It's a simple matter of determining a point in 3D space.

If you had originally asked, "How do I find the x/y coordinates under the mouse for any z-value?," you'd have gotten the same answer.

Once the x/y values are calculated, then you can move your object.

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.