mouse click coords to screen coords
Author
What I am trying to do is take the coordinates from a mouse click, from getCursorPos() and screenToClient(), and convert them to my 3d coordinates, so I can draw a ray into space and check its intersection with various objects. Can someone help with a general way to convert the pixel coordinates to on-screen 3d coordinates ? (the camera can move freely and might be pointing in an arbitrary direction.)
Thanks,
Nitzan
What API are you using?
In DirectX there is a helper function D3DXVec3Unproject.
Call the function two times, with your origin MouseX,MouseY, 0.0f and 1.0f. This will give you two point to form a ray with for intersection tests.
I think for OpenGL there's something similar.
In DirectX there is a helper function D3DXVec3Unproject.
Call the function two times, with your origin MouseX,MouseY, 0.0f and 1.0f. This will give you two point to form a ray with for intersection tests.
I think for OpenGL there's something similar.
When a 3D world coordinate is transformed in to a 2D projected coordinate is loses it's depth. Consequently, it is impossible to transform a 2D coordinate back in to where it was in the world before the transformation.
Generally, what you need to do is use your 2D coordinates to compute a pick ray is screen space. You then need to transform this screen space pick ray by the inverse of your view matrix to get your pick ray origin and direction in world coordinates. You would then use this ray in your intersection tests.
Such a function could look similar to the one below (this is not optomised):
For a little clarification, in the code above m_LevelMesh->GetMesh() just returns a mesh of type ID3DXMesh*. This is an ordinary mesh that is centred around the origin and not transformed when rendered.
If you want to test this computed pick ray against objects placed arbirtarily in your world then you would need to transform the ray origin and direction by the inverse of the object's world position matrix before doing the intersection test with the object. This moves the ray in to the object's body space for testing.
[Edited by - __Daedalus__ on September 14, 2004 5:34:59 AM]
Generally, what you need to do is use your 2D coordinates to compute a pick ray is screen space. You then need to transform this screen space pick ray by the inverse of your view matrix to get your pick ray origin and direction in world coordinates. You would then use this ray in your intersection tests.
Such a function could look similar to the one below (this is not optomised):
D3DXVECTOR3 PickWorld( HWND Hwnd, int ScreenWidth, int ScreenHeight ){ D3DXVECTOR3 vPickRayDir; // Pick ray direction D3DXVECTOR3 vPickRayOrig; // Pick ray origin D3DXVECTOR3 v; // Vector used in computation D3DXMATRIX matProj; // Retrieved projection matrix D3DXMATRIX matView, m; // Retrieved view and computation matrices POINT ptCursor; // Cursor position BOOL bHit; // Was a supplied mesh face hit DWORD dwFace; // The hit face number float fBarry1, fBarry2, fDist; // Barycentric coordinates in face and distance from ray origin // Error check if( m_Graphics->GetDeviceCOM() == NULL ){ return D3DXVECTOR3(0.0f,0.0f,0.0f); } if( m_LevelMesh == NULL ) { return D3DXVECTOR3(0.0f,0.0f,0.0f); } // Get the projection matrix m_Graphics->GetDeviceCOM()->GetTransform( D3DTS_PROJECTION, &matProj ); // Get the cursor position GetCursorPos( &ptCursor ); // Change coordinates relative to the window which the level is rendered in ScreenToClient( Hwnd, &ptCursor ); // Compute the pick ray in screen space v.x = ( ( ( 2.0f * ptCursor.x ) / ScreenWidth ) - 1 ) / matProj._11; v.y = -( ( ( 2.0f * ptCursor.y ) / ScreenHeight ) - 1 ) / matProj._22; v.z = 1.0f; // Get the inverse view matrix m_Graphics->GetDeviceCOM()->GetTransform( D3DTS_VIEW, &matView ); D3DXMatrixInverse( &m, NULL, &matView ); // Transform the screen space pick ray in to 3D coordinates vPickRayDir.x = v.x*m._11 + v.y*m._21 + v.z*m._31; vPickRayDir.y = v.x*m._12 + v.y*m._22 + v.z*m._32; vPickRayDir.z = v.x*m._13 + v.y*m._23 + v.z*m._33; vPickRayOrig.x = m._41; vPickRayOrig.y = m._42; vPickRayOrig.z = m._43; // Collect the hit information D3DXIntersect( m_LevelMesh->GetMesh(), &vPickRayOrig, &vPickRayDir, &bHit, &dwFace, &fBarry1, &fBarry2, &fDist, NULL, NULL ); // If a hit is acquired - store the details if( bHit == TRUE ) { // Normalise direction D3DXVECTOR3 Output, Result; D3DXVec3Normalize( &Output, &vPickRayDir ); // Work out position position Result = vPickRayOrig + (fDist*Output); return Result; } else { return D3DXVECTOR3(0.0f,0.0f,0.0f); }}For a little clarification, in the code above m_LevelMesh->GetMesh() just returns a mesh of type ID3DXMesh*. This is an ordinary mesh that is centred around the origin and not transformed when rendered.
If you want to test this computed pick ray against objects placed arbirtarily in your world then you would need to transform the ray origin and direction by the inverse of the object's world position matrix before doing the intersection test with the object. This moves the ray in to the object's body space for testing.
[Edited by - __Daedalus__ on September 14, 2004 5:34:59 AM]
Quote:Original post by Endurion
What API are you using?
This is the DirectX forum [smile]
Quote:Original post by CoderQuote:Original post by Endurion
What API are you using?
This is the DirectX forum [smile]
Erm... yeah... (slaps myself with a spork)
Author
__Daedalus__,
the code you posted assumes the camera is always pointing in 0,0,1 direction right ?
How do I correctly convert the pixel coordinates if the camera is pointing in some random direction ?
Edit: I actually have another question. Since I am setting the persepctive angle to 45 degrees, why cant I simply figure out the ray by hand ? Shouldnt be too hard to figure out the correct angles at every point (if it's linear). Is it linear ? Is it a correct method to implement picking ?
According to my calculations, if the perspetive is 45 degrees horizontally, it should be 33.69 degrees vertically at 640x480. So at the outer edge, if I rotate the ray by 33.69 in the x axis and 45 degrees in the z axis won't it be pependicular to the screen ? Then I can find any point(s) in between.
Thanks,
Nitzan
[Edited by - nitzan on September 14, 2004 5:47:51 PM]
the code you posted assumes the camera is always pointing in 0,0,1 direction right ?
How do I correctly convert the pixel coordinates if the camera is pointing in some random direction ?
Edit: I actually have another question. Since I am setting the persepctive angle to 45 degrees, why cant I simply figure out the ray by hand ? Shouldnt be too hard to figure out the correct angles at every point (if it's linear). Is it linear ? Is it a correct method to implement picking ?
According to my calculations, if the perspetive is 45 degrees horizontally, it should be 33.69 degrees vertically at 640x480. So at the outer edge, if I rotate the ray by 33.69 in the x axis and 45 degrees in the z axis won't it be pependicular to the screen ? Then I can find any point(s) in between.
Thanks,
Nitzan
[Edited by - nitzan on September 14, 2004 5:47:51 PM]
Quote:Original post by nitzanThe camera is pointing in any random direction. The camera's arbitrary translation, and rotation values are stored in the view matrix which you retrieve each time in the above function by calling Device->GetTransform( D3DTS_VIEW, &matView ).
__Daedalus__,
the code you posted assumes the camera is always pointing in 0,0,1 direction right ?
How do I correctly convert the pixel coordinates if the camera is pointing in some random direction ?
Edit: I actually have another question. Since I am setting the persepctive angle to 45 degrees, why cant I simply figure out the ray by hand ? Shouldnt be too hard to figure out the correct angles at every point (if it's linear). Is it linear ? Is it a correct method to implement picking ?
Thanks,
Nitzan
Doing these calculations, the pick ray origin is the same position as the the camera (e.g. view matrix) in the world. And the ray direction is the screen space pick ray (a 3D vector built using your mouse coordinates and the projection matrix) transformed in relation to the (inverse) view matrix.
Author
So I implemented the code you posted Daedalus, up to D3DXIntersect. It seems to work fine but I dont understand something. vPickRayOrig always ends up being the camera location, meaning the ray seems to eminate from the center of the screen instead of where I actaully clicked. Also vPickRayDir is a 2 dimensional vector (its Z value is always zero) pointing from the center of the screen toward the point of click. So what I am guessing is that D3DXIntersect does some more work. Any idea how to get the exact point of click and the correct direction the pick ray is traveling at (I am using my own intersection routine with my own model format) ?
Once again:
vPickRayDir seems to be a 2d vector originating at the center of the screen and pointing toward the direction of the mouse click.
vPickRayOrig is always the camera location, meaning the center of the screen.
Thanks!
Nitzan
Once again:
vPickRayDir seems to be a 2d vector originating at the center of the screen and pointing toward the direction of the mouse click.
vPickRayOrig is always the camera location, meaning the center of the screen.
Thanks!
Nitzan
that is right nitzan. the origin of the ray should be from the cameras location. I had the same problems recently with picking so i know what its like. i came up with a simple analogy to help understand it.
Look out of a window. the glass in the window is your screen, your eye is the camera and everything past the glass is you world. your finger acts as your mouse cursor.
now, choose something you want to pick. (preferable towards the left edge of the window to highlight my point better) say a tree/fencepost/car etc.
now, put your finger on the glass where you see the object you chose. now, your finger should be fully or partly obscuring the object. Now keep you finger there, and move your head to looking at your finger at a right angle to the glass. you will see that you finger (read cursor) is only about 50-100 pixels away from the screen edge.
but, the object you picked is far past the screens area to the left. (lets say, in screen coordinates (ie, relative to the glass/screen) -1000 pixels)
this is why we need to do all the ray stuff, because the object isnt really there, its just where we see it.
when your head, finger and object were in line, your basically projecting a ray from your eye (camera) through your finger (cursor).
another analogy, is with the perspective. to mimic a lower perspective, move your eye further from the window, and vice versa with a higher perspective.
hope that helps you understand. but what is your aim for this? are you trying to select (pick) something in your world? if you are then i suggest using 'bounding volumes'. This way you do your 'checking for a hit' with the bounding volume, and not the mesh (as in daedalus's code above). This is more efficient, but once you have determined it 'hit' the bounding volume, you can then go onto checking which part of the mesh it hit. but i suspect you wont be needing to do this.
Look out of a window. the glass in the window is your screen, your eye is the camera and everything past the glass is you world. your finger acts as your mouse cursor.
now, choose something you want to pick. (preferable towards the left edge of the window to highlight my point better) say a tree/fencepost/car etc.
now, put your finger on the glass where you see the object you chose. now, your finger should be fully or partly obscuring the object. Now keep you finger there, and move your head to looking at your finger at a right angle to the glass. you will see that you finger (read cursor) is only about 50-100 pixels away from the screen edge.
but, the object you picked is far past the screens area to the left. (lets say, in screen coordinates (ie, relative to the glass/screen) -1000 pixels)
this is why we need to do all the ray stuff, because the object isnt really there, its just where we see it.
when your head, finger and object were in line, your basically projecting a ray from your eye (camera) through your finger (cursor).
another analogy, is with the perspective. to mimic a lower perspective, move your eye further from the window, and vice versa with a higher perspective.
hope that helps you understand. but what is your aim for this? are you trying to select (pick) something in your world? if you are then i suggest using 'bounding volumes'. This way you do your 'checking for a hit' with the bounding volume, and not the mesh (as in daedalus's code above). This is more efficient, but once you have determined it 'hit' the bounding volume, you can then go onto checking which part of the mesh it hit. but i suspect you wont be needing to do this.
Check out the directx Pick sample...cut and paste the code...you're done.
Seems like somebody posts the same question every day or two...and I was guilty of it myself once. MS provided a lot of fairly concise samples with DirectX that illustrate a lot of the concepts that people commonly use, and they included a SampleBrowser app to make samples easy to find. For all of the complaining that everybody does (including myself) about MS's documentation, they made up for that by providing public domain code that can often be pasted right into your program. I think their logic was that it's easier step through the code to see what's going on than to read some really confusing tutorial. We can save ourselves a lot of wheel-reinventing by following the breadcrumbs that Uncle Bill left for us.
Seems like somebody posts the same question every day or two...and I was guilty of it myself once. MS provided a lot of fairly concise samples with DirectX that illustrate a lot of the concepts that people commonly use, and they included a SampleBrowser app to make samples easy to find. For all of the complaining that everybody does (including myself) about MS's documentation, they made up for that by providing public domain code that can often be pasted right into your program. I think their logic was that it's easier step through the code to see what's going on than to read some really confusing tutorial. We can save ourselves a lot of wheel-reinventing by following the breadcrumbs that Uncle Bill left for us.
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement