Screen to World Transformation with Ortho Projection

Started by
8 comments, last by mmor420 21 years, 11 months ago
Hi all, I have been doing some major research on this topic, trying to convert a screen point to world coordinate using an orthogonal projection matrix, altho I have been unsuccessful. Hopefully someone here has more insight than I on this subject. The DXSDK uses this formula to convert a screen point to view coordinates with a Projection Matrix:
  
// Compute the vector of the pick ray in screen space

D3DXVECTOR3 v;
v.x =  ( ( ( 2.0f * ptCursor.x ) / m_d3dsdBackBuffer.Width  ) - 1 ) / matProj._11;
v.y = -( ( ( 2.0f * ptCursor.y ) / m_d3dsdBackBuffer.Height ) - 1 ) / matProj._22;
v.z =  1.0f;
   
then to convert to world:
  
// Get the inverse view matrix

D3DXMATRIX matView, m;
m_pd3dDevice->GetTransform( D3DTS_VIEW, &matView );
D3DXMatrixInverse( &m, NULL, &matView );

// Transform the screen space pick ray into 3D space

vPickRayDir.x  = v.x*m._11 + v.y*m._21 + v.z*m._31;
vPickRayDir.y  = v.x*m._12 + v.y*m._22 + v.z*m._32;
vPickRayDir.z  = v.x*m._13 + v.y*m._23 + v.z*m._33;       
vPickRayOrig.x = m._41;
vPickRayOrig.y = m._42;
vPickRayOrig.z = m._43;
   
but this does not produce correct results when using an orthogonal matrix. I am at a complete loss on this. Could anyone here be of any assistance? Maybe it has something to do with the matrix determinant? (altho i have no clue ) Thanks, Zygote- [edited by - mmor420 on May 6, 2002 3:42:32 PM]
Advertisement

Hi!

I haven''t analyzed the code,

but, in general:

You already said it: coordinates are projected
onto the screen. With loss of one coordinate.
You can''t derive "the" world point from screen coordinates,
because there are an infinite number of coordinates
that project into any of the screen pixels.
Imagine all those light rays, which fall from the world
onto your camera. Points that are on such a ray, but at a different distance, will all project into the same screen pixel.
Or, to fit your case better, you could just think of aprojector, which has<br>a 2D image as source. You can put your white projection wall<br>(or whatever) at an infinite number of different distances, but you will always see the same picture. <br>(at different size, though, for, no orthographic projection here.)<br><br><br>Tell us what you are going to do, and let''s see how we<br>can help.<br><br> - unshaven bastard<br><br><br> </b>
Thanks For the reply Guess i wasnt clear enough.

What i''m trying to do, is about the same that i''ve seen numerous posts here before: "cast a ray from the camera location twards a point on the screen that the mouse is over" IE: "click on a world coordinate"

what i have done, is opened the Pick example in the DXSDK and created a 10x10 grid of lines. In the "Pick" function, after getting the "world vector" from the point clicked on screen, I have it displaying the world coordinate relative to the Y axis:


  D3DXPLANE p;D3DXPlaneFromPointNormal(&p,&D3DXVECTOR3(0,0,0),&D3DXVECTOR3(0,1,0));D3DXVECTOR3 vec = vPickRayOrig - vPickRayDir;D3DXPlaneIntersectLine(&g_mouse,&p,&vPickRayOrig,&vec);  


see the first post for getting the vPickRayOrig and vPickRayDir

using a perspective viewport, i am able to retrieve the correct world coordinates from any spot clicked on-screen. (verified by the grid)

however, when i switch to an orthogonal projection matrix, it seems the very principles wich make the perspective transformation code work properly do not apply to an orthogonal view.

:D

So:

The code i have put here is straight from the DXSDK, it works great and all that.. (with a perspective viewport).. all i''m trying to do is get it to work with an orthogonal viewport. It should only require very basic changes (??) altho it seems nobody really knows for shure how to do this (every reply i''ve seen to such a request has been beaten around the bush with silly replys.. or have gotten some really off the wall ''i''m smarter than u'' replys that do nobody any good.)

this is very basic stuff.. altho i am definately no expert in the field. I am not looking for a long mathematical jargon speech on how the universe spins, but just a simple answer to a simple question.


Thanks,
Zygote-
[bump]
See D3DXVec3Unproject.

xyzzy
D3DXVec3Unproject DOES NOT WORK! at least i have been unsuccessful in using it. Possibly from the lack of examples... the function looks so easy, just pass in the viewport,projection,view,world matrices and voila! an unprojected point... but its not so.. Have u tried using that function?? I''d guess not. but if you have any helpful info on how to make it work properly i''m all ears.

Zygote-
Sorry for my ignorance
I did get it to work..


  //get the mouse position relative to the viewport      POINT ptCursor;      GetCursorPos( &ptCursor );      ScreenToClient( m_hWnd, &ptCursor );      D3DXMATRIX matProj;      m_pd3dDevice->GetTransform( D3DTS_PROJECTION, &matProj );      D3DVIEWPORT8 viewport;      m_pd3dDevice->GetViewport(&viewport);      // Get the inverse view matrix      D3DXMATRIX matView;      D3DXMATRIX matViewInv;      m_pd3dDevice->GetTransform( D3DTS_VIEW, &matView );      //unproject the point      D3DXVECTOR3 mousepos = D3DXVECTOR3(ptCursor.x,ptCursor.y,1.0f);      D3DXVECTOR3 unproj;      D3DXVec3Unproject(&unproj,&mousepos,&viewport,&matProj,&matView,NULL);      D3DXVECTOR3 vPickRayDir;      D3DXVECTOR3 vPickRayOrig;      vPickRayDir = unproj;      //get the camera location      D3DXMatrixInverse( &matViewInv, NULL, &matView );      vPickRayOrig.x = matViewInv._41;      vPickRayOrig.y = matViewInv._42;      vPickRayOrig.z = matViewInv._43;      //get the world coordinates relative to the X,Z axis      D3DXPLANE p;      D3DXPlaneFromPointNormal(&p,&D3DXVECTOR3(0,0,0),&D3DXVECTOR3(0,1,0));      D3DXPlaneIntersectLine(&g_mouse,&p,&vPickRayOrig,&vPickRayDir);  
I take that back.. i was using perspective projection matrix.. The Unproject function function does not work with an orthogonal projection matrix

Any more ideas? or maybe have a clue why D3DXVec3Unproject does not work with an orthogonal projection matrix??

thanks,
Zygote-
I had similar difficulty. This code was for a game with an isometric view, but a completely 3d rendered. This function takes the given screen coordinates and calculates where on the world space plane z = 0 the ray from eye point to z = 0 is. This code works whether using an orthogonal view point or not but it does do things slightly different.


  /*////////////////////////////////////////////////////////////////////////////	NAME: ScreenXYToWorldZPlane	PURPOSE: Find the XY coordinates in world space at z = 0 of a ray cast into			 the screen at specified screen coordinates	INPUT: ScreenX & ScreenY - screen coordinates		   OutX & OutY - calculated world coordinates                 	RETURNS: none/////////////////////////////////////////////////////////////////////////////*/

void BattleInterface::ScreenXYToWorldZPlane(SHORT ScreenX, SHORT ScreenY, float *OutX, float *OutY)
{

D3DXMATRIX w, v, p;
D3DXMatrixIdentity(&w);

Screen->GetD3DDevice()->GetTransform( D3DTS_VIEW, &v );
Screen->GetD3DDevice()->GetTransform( D3DTS_PROJECTION, &p );

D3DVIEWPORT8 viewport;
Screen->GetD3DDevice()->GetViewport(&viewport);

D3DXVECTOR3 in, RayPos, RayPos2, Intersect;
// Get the inverse view matrix (eye position)

D3DXMATRIX iv;
D3DXMatrixInverse( &iv, NULL, &v );

in.x = iv._41;
in.y = iv._42;
in.z = iv._43;

D3DXVec3Unproject(&RayPos, &in, &viewport, &p, &v, &w);

in.x = ScreenX;
in.y = ScreenY;
if( Screen->GetCamera()->Orthogonal )
{
in.z = 0.0f;
}
else
{
in.z = 1.0f;
}

D3DXVec3Unproject(&RayPos2, &in, &viewport, &p, &v, &w);

D3DXPLANE plane;
D3DXPlaneFromPointNormal( &plane, &D3DXVECTOR3(0,0, 0), &D3DXVECTOR3(0,0,1) );
D3DXPlaneIntersectLine(&Intersect, &plane, &RayPos, &RayPos2);

*OutX = Intersect.x;
*OutY = Intersect.y;
}


Hope it helps.

Jack
Well it does come kinda close to working, but still no cigar..

Test it by creating a 10x10 grid, and check the mouse position as it goes around the grid.. it comes up very inaccurate.

with my camera facing the origin (0,0,0) at location: (8,10,4) with an up vector of (0,1,0) it works good in perspective projection.

perspective projection:
D3DXMatrixPerspectiveFovLH( &matProj, D3DX_PI/4, fAspect, 1.0f, 100.0f );  


however, if i change the camera location to be (0,10,4) with up vector of (0,1,0) the coordinates do not come out correctly any more.

when i hold the mouse over the grid line at ''1'', it says i have intersected at .5 ..

how does the camera location affect the outcome of this function so much? this is in perspective projection!

in orthogonal projection at the same camera position it says: 18.9,0,17.27 at (1,0,0) this is obviously unsatisfactory.

i dont know how you have your view matrix set up, but i can assure you if you change your camera''s (view matrix) location your coordinates will not come up correct.


i have attatched the code i used below. it is basically a stripped "Pick" example that where i have removed the mesh and put in a 10x10 grid centered at the origin. I am calling D3DXPlaneIntersectsLine to get the world coordinates from the "pick ray".

What Gives?

I know i''m not messing this stuff up that bad.. There is definately something wrong with D3DXVec3Unproject, for it doesnt even work when the camera is at different angles.

any more ideas? There must be a way to do this correctly.. they got projected to the screen somehow..



Zygote-





  #define STRICT#include <math.h>#include <stdio.h>#include <D3DX8.h>#include "D3DApp.h"#include "D3DFont.h"#include "D3DFile.h"#include "D3DUtil.h"#include "DXUtil.h"#include "Resource.h"bool g_ortho = false;D3DXVECTOR3 g_mouse;D3DXVECTOR3 g_cam;struct lineVert{   D3DXVECTOR3 p;   D3DCOLOR    color;};#define FVF_LINE ( D3DFVF_XYZ | D3DFVF_DIFFUSE )//-----------------------------------------------------------------------------// Name: Pick()// Desc: Checks if mouse point hits geometry//       the scene.//-----------------------------------------------------------------------------HRESULT CMyD3DApplication::Pick(){   POINT ptCursor2;   GetCursorPos( &ptCursor2 );   ScreenToClient( m_hWnd, &ptCursor2 );   D3DVIEWPORT8 viewport;	   m_pd3dDevice->GetViewport(&viewport);	   SHORT ScreenX = ptCursor2.x;   SHORT ScreenY = ptCursor2.y;   D3DXMATRIX w, v, p;	   D3DXMatrixIdentity(&w);	   m_pd3dDevice->GetTransform( D3DTS_VIEW, &v );	   m_pd3dDevice->GetTransform( D3DTS_PROJECTION, &p );	      D3DXVECTOR3 in, RayPos, RayPos2, Intersect;		   // Get the inverse view matrix (eye position)	   D3DXMATRIX iv;	   D3DXMatrixInverse( &iv, NULL, &v );	   in.x = iv._41;	   in.y = iv._42;	   in.z = iv._43;	   D3DXVec3Unproject(&RayPos, &in, &viewport, &p, &v, &w);	   //RayPos = in;   in.x = ScreenX;	   in.y = ScreenY;	   if( g_ortho )	   {		      in.z = 0.0f;	   }	   else	   {		      in.z = 1.0f;	   }      D3DXVec3Unproject(&RayPos2, &in, &viewport, &p, &v, &w);	   D3DXPLANE plane;	   D3DXPlaneFromPointNormal( &plane, &D3DXVECTOR3(0,0, 0), &D3DXVECTOR3(0,1,0) );	      D3DXPlaneIntersectLine(&Intersect, &plane, &RayPos, &RayPos2);	   g_mouse = Intersect;   return 1;   }//-----------------------------------------------------------------------------// Name: FrameMove()// Desc: Called once per frame, the call is the entry point for animating//       the scene.//-----------------------------------------------------------------------------HRESULT CMyD3DApplication::FrameMove(){    // Rotate the camera about the y-axis    D3DXVECTOR3 vFromPt   = D3DXVECTOR3( 0.0f, 0.0f, 0.0f );    D3DXVECTOR3 vLookatPt = D3DXVECTOR3( 0.0f, 0.0f, 0.0f );    D3DXVECTOR3 vUpVec    = D3DXVECTOR3( 0.0f, 1.0f, 0.0f );    vFromPt.x = 8.0f;    vFromPt.y = 10.0f;    vFromPt.z = 8.0f;    D3DXMATRIX matView;    D3DXMatrixLookAtLH( &matView, &vFromPt, &vLookatPt, &vUpVec );    m_pd3dDevice->SetTransform( D3DTS_VIEW, &matView );    return S_OK;}//-----------------------------------------------------------------------------// Name: Render()// Desc: Called once per frame, the call is the entry point for 3d//       rendering. This function sets up render states, clears the//       viewport, and renders the scene.//-----------------------------------------------------------------------------HRESULT CMyD3DApplication::Render(){    // Set up the cursor    POINT ptCursor;    GetCursorPos( &ptCursor );    ScreenToClient( m_hWnd, &ptCursor );    m_pd3dDevice->SetCursorPosition( ptCursor.x, ptCursor.y, 0L );    // Check for picked triangles    Pick();    // Clear the viewport    m_pd3dDevice->Clear( 0L, NULL, D3DCLEAR_TARGET|D3DCLEAR_ZBUFFER,                         0x000000ff, 1.0f, 0L );    // Begin the scene    if( SUCCEEDED( m_pd3dDevice->BeginScene() ) )    {        // Set render mode to lit, solid triangles        m_pd3dDevice->SetRenderState( D3DRS_FILLMODE, D3DFILL_SOLID );        m_pd3dDevice->SetRenderState( D3DRS_LIGHTING, TRUE );        // If a triangle is picked, draw it        if( m_dwNumIntersections > 0 )        {            // Draw the picked triangle            m_pd3dDevice->SetVertexShader( D3DFVF_VERTEX );            m_pd3dDevice->SetStreamSource( 0, m_pVB, sizeof(D3DVERTEX) );            m_pd3dDevice->DrawPrimitive( D3DPT_TRIANGLELIST, 0, m_dwNumIntersections );            // Set render mode to unlit, wireframe triangles            m_pd3dDevice->SetRenderState( D3DRS_FILLMODE, D3DFILL_WIREFRAME );            m_pd3dDevice->SetRenderState( D3DRS_LIGHTING, FALSE );        }        // Set render mode to unlit, wireframe triangles            m_pd3dDevice->SetRenderState( D3DRS_FILLMODE, D3DFILL_WIREFRAME );            m_pd3dDevice->SetRenderState( D3DRS_LIGHTING, FALSE );        int width = 10;        for(int x=-10;x<11;x++)        {           lineVert v[2];           if(x == 0)           {              v[0].color = D3DCOLOR_XRGB(0,0,0);              v[1].color = D3DCOLOR_XRGB(0,0,0);           }           else if(x % 2 == 0)           {              v[0].color = D3DCOLOR_XRGB(100,100,100);              v[1].color = D3DCOLOR_XRGB(100,100,100);           }           else           {              v[0].color = D3DCOLOR_XRGB(255,0,0);              v[1].color = D3DCOLOR_XRGB(255,0,0);           }           v[0].p.x = x;           v[0].p.y = 0;           v[0].p.z = -width;           v[1].p.x = x;           v[1].p.y = 0;           v[1].p.z = width;           m_pd3dDevice->SetVertexShader(FVF_LINE);           m_pd3dDevice->DrawPrimitiveUP(D3DPT_LINELIST,1,(void*)v,sizeof(lineVert));           v[0].p.x = -width;           v[0].p.y = 0;           v[0].p.z = x;           v[1].p.x = width;           v[1].p.y = 0;           v[1].p.z = x;           m_pd3dDevice->DrawPrimitiveUP(D3DPT_LINELIST,1,(void*)v,sizeof(lineVert));        }        // Render the mesh        m_pObject->Render( m_pd3dDevice );        // Output statistics        char buff[255];        sprintf(buff,"mouse world: (%.2f,%.2f,%.2f)",g_mouse.x,g_mouse.y,g_mouse.z);        m_pFont->DrawText( 2,  0, D3DCOLOR_ARGB(255,255,255,0), buff);        sprintf(buff,"camera loc: (%.2f,%.2f,%.2f)",g_cam.x,g_cam.y,g_cam.z);        m_pFont->DrawText( 2,  20, D3DCOLOR_ARGB(255,255,255,0), buff);        // End the scene.        m_pd3dDevice->EndScene();    }    return S_OK;}//-----------------------------------------------------------------------------// Name: RestoreDeviceObjects()// Desc: Initialize scene objects.//-----------------------------------------------------------------------------HRESULT CMyD3DApplication::RestoreDeviceObjects(){    m_pFont->RestoreDeviceObjects();    m_pFontSmall->RestoreDeviceObjects();    // Set miscellaneous render states    m_pd3dDevice->SetRenderState( D3DRS_DITHERENABLE,   FALSE );    m_pd3dDevice->SetRenderState( D3DRS_SPECULARENABLE, FALSE );    m_pd3dDevice->SetRenderState( D3DRS_ZENABLE,        TRUE );    m_pd3dDevice->SetRenderState( D3DRS_AMBIENT,        0x00444444 );    // Set the world matrix    D3DXMATRIX matIdentity;    D3DXMatrixIdentity( &matIdentity );    m_pd3dDevice->SetTransform( D3DTS_WORLD,  &matIdentity );    // Set the projection matrix    D3DXMATRIX matProj;    FLOAT fAspect = ((FLOAT)m_d3dsdBackBuffer.Width) / m_d3dsdBackBuffer.Height;    if(GetKeyState(VK_SPACE) & 0x80)       g_ortho = true;    else       g_ortho = false;    if(g_ortho)    {       D3DXMatrixOrthoLH(&matProj,40,40,1,100);    }    else    {       D3DXMatrixPerspectiveFovLH( &matProj, D3DX_PI/4, fAspect, 1.0f, 100.0f );     }    m_pd3dDevice->SetTransform( D3DTS_PROJECTION, &matProj );    // Setup a material    D3DMATERIAL8 mtrl;    D3DUtil_InitMaterial( mtrl, 1.0f, 1.0f, 1.0f, 1.0f );    m_pd3dDevice->SetMaterial( &mtrl );    // Set up lighting states    D3DLIGHT8 light;    D3DUtil_InitLight( light, D3DLIGHT_DIRECTIONAL, 0.1f, -1.0f, 0.1f );    m_pd3dDevice->SetLight( 0, &light );    m_pd3dDevice->LightEnable( 0, TRUE );    m_pd3dDevice->SetRenderState( D3DRS_LIGHTING, TRUE );    return S_OK;}  



This topic is closed to new replies.

Advertisement