Sign in to follow this  
Duncanf

Mouse picking slightly inaccurate

Recommended Posts

Duncanf    133
Ello all, I've got some problems with picking, I've got it working but all my picks are slightly off target. The picking seems to work perfectly from the window coordinates 0,0 but as soon as I try anywhere else the actual location of the pointer and the location of the ray hit are increasingly wrong - worst in the bottom right of the window. Here's the code I'm using:
//SetCapture called in message handler
if( GetCapture() )
{
POINT ptCursor;
GetCursorPos( &ptCursor );
ScreenToClient( windowHandle, &ptCursor );

D3DXVECTOR3 v;
v.x =  ( ( ( 2.0f * ptCursor.x ) / winWidth  ) - 1 ) / projection._11;
v.y = -( ( ( 2.0f * ptCursor.y ) / winHeight ) - 1 ) / projection._22;
v.z =  1.0f;

D3DXMATRIX m;
D3DXMatrixInverse( &m, NULL, &view );

rayFromMouse.x  = v.x*m._11 + v.y*m._21 + v.z*m._31;
rayFromMouse.y  = v.x*m._12 + v.y*m._22 + v.z*m._32;
rayFromMouse.z  = v.x*m._13 + v.y*m._23 + v.z*m._33;
D3DXVec3Normalize(&rayFromMouse,&rayFromMouse);
rayFromMousePos = camera->getPosition();

SetCapture(0);
}

Does anyone know what I'm doing wrong? Cheers for any help, Duncan

Share this post


Link to post
Share on other sites
Mike Bossy    697
This happend to me when my client space is smaller than I thought.

Repro of problem:

1. Window was 640x480.
2. Client area of window is slightly smaller than 640x480 due to title bar, etc.
3. Created D3D surface using the default values for the window. (which uses the client area size).
4. My mouse positioning calculations used 640x480 ans the size of the surface and not the actual client size.

Solution:
In step 3 I created the D3D surface to be of the proper size, 640x480, and everything lined up again.

Share this post


Link to post
Share on other sites
DarkZoulz    106
Are you running in fullscreen or windowed mode? If your running in windowed mode, are you sure the actual drawing area is what you think it is? I mean the window frame and titlebar etc takes away some of the actual drawing space. That might cause a small offset.

This was my problem, and took about a week to finally figure out. doh.

Share this post


Link to post
Share on other sites
Duncanf    133
Cheers for the replies, I've tried getting the exact (windowed) viewport size by getting the backbuffer and using the width and height from there - but it's still inaccurate. How are you getting your values of width and height?

Share this post


Link to post
Share on other sites
Duncanf    133
Thanks, that works perfectly. For anyone who searches the forum and needs a code sample:


RECT rect = { 0, 0, eng->winWidth, eng->winHeight };
if (eng->windowed) AdjustWindowRectEx(&rect, WS_CAPTION | WS_SYSMENU, false, WS_EX_APPWINDOW);

//try to create a window using class
eng->windowHandle = ::CreateWindow("DuncEngine", title, // class name, title
WS_EX_TOPMOST, //style
0, 0, rect.right-rect.left, rect.bottom-rect.top, //x,y,w,h
0, //no parent
0, //no menu
instance, //instance given
0); //no extra mem

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this