hi guys, i'm working on my simple 3d modeller program, and i'm now implementing the mouse pick function, but i cannot setup the pick matrix correctly, here is current code,

void Camera::setCameraForSelection(size_t x1,size_t y1,size_t x2,size_t y2)
{
glViewport((GLint)startX,(GLint)startY,(GLint)width,(GLint)height);
GLint viewport[4];
glGetIntegerv (GL_VIEWPORT, viewport);
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
gluPickMatrix((GLdouble)x1,(GLdouble)(viewport[3]-y1),(GLdouble)(x2-x1),(GLdouble)( y2-y1),viewport);
gluPerspective(angle,width/height,nearPlane,farPlane);
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
gluLookAt(eye.x,eye.y,eye.z,target.x,target.y,target.z,up.x,up.y,up.z);
};

the (x1,y1) is the up-left point of the selection area and the (x2,y2) is the low-right point of the area. but when i do selection like this, nothing happened: and when i select in empty area, some faces are selected: i found an example in the opengl redbook, the difference is that the example pick up things when the mouse clicks, so it setups the pick matrix like this: gluPickMatrix((GLdouble) x,(GLdouble) (viewport[3] - y), 5.0, 5.0, viewport); since i need area selection feature, i modified the pick matrix to: gluPickMatrix((GLdouble) x1,(GLdouble) (viewport[3] - y1),(GLdouble)(x2-x1),(GLdouble)( y2-y1), viewport); i do not see any problem, but it just doesn't work correctly. and if i use the example code and change my code to mouse-click pick, it works perfectly. so please help me. thank you very much. [Edited by - billconan on September 7, 2006 2:03:41 PM]

Quote:Original post by remdul
I'm assuming you know that [0,0] lies at the bottom left corner (the Redbook sample takes that in account, not sure you are aware of that) and you have to flip the cursor Y coordinate.

This is a nice trick that I used (from OpenGL FAQ):

Quote:20.040 How can I debug my picking code?

A good technique for debugging picking or selection code is not to call glRenderMode(GL_SELECT). Simply comment out this function call in your code. The result is instead of performing a selection, your code will render the contents of the pick box to your window. This allows you to see visually what is inside your pick box.

Along with this method, it's generally a good idea to enlarge your pick box, so you can see more in your window.

Hope that helps.

Sweet GUI by the way! :)

thanks, i thought of render the result on the screen, but the swap buffer function is not in the same class, i will try it anyway. thanks.

hi there, i've test the selection area by render it out, but i just don't get it. the mouse position is correctly pasd to the pick matrix function, however, the selection area seems to offset up-left. i just don't get it.

the blue area is the selection i made and the red one is the pick matrix function generated:



this is the scene with the pick matix applied:



i still cannot get it. why?

haha, i find out the problem, the first two parameters of the function gluPickMatrix is not the coordinates of the up-left corner of the selection area as i ever assumed. however they are the center point's coordinates of the selection area.

so i changed the code to

gluPickMatrix((GLdouble) (x1+x2)/2,(GLdouble) (viewport[3] - (y1+y2)/2), (GLdouble)(x2-x1),(GLdouble)( y2-y1), viewport);

and it works!!!! :)