Sign in to follow this  
Icetigris

Picking problems - bad return values

Recommended Posts

I'm trying to select a cube by clicking on it using GL_SELECT render mode type selection, but it keeps returning -1 or 0 arbitrarily. I've been told to write my own picking code, but I'm not really sure how to do it. I am using jogl in CentOS 5 (RedHat). Here's my picking function:
public void pick(GL gl)
    {
        if(mouseLeftClicked)
        {
            System.out.println("you clicked your mouse and are attempting to pick something");
            IntBuffer selectionBuffer = BufferUtil.newIntBuffer(BUFSIZE);
            int hits = 0;
            IntBuffer viewport = BufferUtil.newIntBuffer(4);        
        
            gl.glGetIntegerv(GL.GL_VIEWPORT, viewport); //setup viewport
            gl.glSelectBuffer(hits, selectionBuffer); //setup hits buffer
            
            gl.glRenderMode(GL.GL_SELECT); //put gl context into selection mode
            gl.glInitNames(); //initialise name stack
            gl.glPushName(0); //push crap name onto stack to avoid error
            gl.glPopName(); //pop crap name off name stack
            
            gl.glMatrixMode(GL.GL_PROJECTION);
            gl.glPushMatrix();
            
                gl.glLoadIdentity();
                glu.gluPickMatrix(mouseX, viewport.get(3) - mouseY, pickWidth, pickHeight, viewport);
                glu.gluPerspective(45.0f, ((float)winWidth/(float)winHeight), 1.0, 50.0);        
                
                gl.glMatrixMode(GL.GL_MODELVIEW);
                
                cube.drawCubes(gl);
                
                gl.glMatrixMode(GL.GL_PROJECTION);
                
            gl.glPopMatrix();
            
            gl.glMatrixMode(GL.GL_MODELVIEW);
            
            gl.glFlush();
            hits = gl.glRenderMode(GL.GL_RENDER);
            
            processHits(hits, selectionBuffer);
            
            mouseLeftClicked = false;
            System.out.println("oh we got to the end of picking; selectionBuffer has " + selectionBuffer.capacity() + " elements");
        }
    }

Share this post


Link to post
Share on other sites
I actually recently posted a similar problem, but figured out that I was getting -1 because my pickbuffer size was too small.

As far as the 0 result, you might try temporarily commenting out glRenderMode calls, so basically you aren't actually doing picking, but you will get to see your picking window get drawn. This is a good way to ensure that your selection area is correct.

If your selection area looks correct and your pick buffer is sufficiently large, I would guess your problem lies in the way you do naming.

Share this post


Link to post
Share on other sites
I think it might be a problem with naming, but I'm not sure. When I commented out the glRenderMode(GL_SELECT) and clicked around, colours from the wrong parts of my rainbow debug cube flashed. I'm not sure why they would do that unless it's a translation or z-buffer issue. I also don't think my pick buffer is too small, since there are 54 cubes on screen and the buffer is 512 elements in size.

The way I have it set up is that I have a class called CubeFactory which makes a list of Cube objects, each of which have their own name (i.e.
int name
). Each cube draws itself and so does glLoadName when drawn in selection mode.

Here is the draw function in Cube:
 /**
* Draws a single cube
* @param gl
*/
public void draw(GL gl)
{
gl.glPushMatrix();
gl.glTranslatef(location.getX(), location.getY(), location.getZ());

if(amISelected)
{//have to put a glBegin and glEnd in here
gl.glColor4f(1.0f, 1.0f, 1.0f, 1.0f);
glut.glutWireCube(size + 0.1f);

System.out.println("You picked cube #" + name);
}

//ONLY WORKS WHEN IN GL_SELECT RENDER MODE
gl.glLoadName(name);


gl.glEnableClientState(gl.GL_VERTEX_ARRAY);
gl.glEnableClientState(gl.GL_COLOR_ARRAY);
gl.glVertexPointer(3, gl.GL_FLOAT, 0, vertexBuffer);
gl.glColorPointer(4, gl.GL_FLOAT, 0, colorBuffer);

gl.glDrawElements(gl.GL_TRIANGLES, 36, gl.GL_UNSIGNED_INT, indicies);

gl.glDisableClientState(gl.GL_COLOR_ARRAY);
gl.glDisableClientState(gl.GL_VERTEX_ARRAY);


gl.glPopMatrix();

}

Share this post


Link to post
Share on other sites
First off, the pickbuffer size can be deceiving. Each "hit" in the pickbuffer takes up more than one byte. It takes up...4 ints at the least (16 bytes), if I remember correctly. I suppose the point is it doesn't really hurt to expand the buffer and try. I ended up expanding mine up to 4096 before I got rid of the -1 return.

Something to try - instead of using LoadName, use PushName and PopName. If you never call PushName and you use LoadName, a GL error will be set because there's nothing on the name stack. I don't use LoadName at all, but rather Push and Pop names off. If you want to use LoadName, make sure you've loaded at least one name onto the stack using PushName before calling LoadName.

Share this post


Link to post
Share on other sites
I tried using glPushName and glPopName inititally and when I put them back nothing changed. I expanded the pick buffer to 4096 and I'm still getting -1 returns, even with 1 cube being drawn.

Share this post


Link to post
Share on other sites
I'm looking at your code, and I think this might be a problem:

gl.glPushName(0); //push crap name onto stack to avoid error
gl.glPopName(); //pop crap name off name stack

The purpose of Pushing a "crap name" onto the stack is to avoid calling glLoadName with an empty stack. So pushing a name onto the stack and immediately popping it off doesn't do you any good. Are you polling glError somewhere to make sure there's nothing else going wrong?

If those avenues don't provide you with any answers, I have one last idea you might try. Do picking on the whole screen. First, make sure you aren't messing up the matrix calculations somewhere by commenting out glRenderMode calls and drawing the picking window, only this time your call to gluPickMatrix will look something like this:

glu.gluPickMatrix(winWidth/2, winHeight/2, winWidth, winHeight, viewport);

It will be really obvious if there's a problem with, say, your modelview matrix, because you'll see that there are two different scenes being drawn.

Once you've confirmed that the scenes being drawn are the same, go ahead and run actual picking on the entire scene, and see what your results are. If you're getting -1, don't hesitate to increase your pickbuffer size to epic proportions (32768 even).

If that doesn't work, I don't have any clue where the problem lies.

Share this post


Link to post
Share on other sites
I expanded the pick buffer to the size you said, checked to see if it was drawing improperly (it is drawing fine; I checked by commenting out glRenderMode(GL_SELECT) and only drawing the scene on click), and commented out that first glPopName. I'm not sure if I'm using glGetError() right, but that returned null.

Here is my current pick function:
public void pick(GL gl)
{
if(mouseLeftClicked)
{
System.out.println("you clicked your mouse and are attempting to pick something");
IntBuffer selectionBuffer = BufferUtil.newIntBuffer(BUFSIZE);
int hits = 0;
double[] worldCoords = new double[3];
IntBuffer viewport = BufferUtil.newIntBuffer(4);
DoubleBuffer modelview = BufferUtil.newDoubleBuffer(16);
DoubleBuffer projection = BufferUtil.newDoubleBuffer(16);

DoubleBuffer objectPosition = BufferUtil.newDoubleBuffer(3);

gl.glGetIntegerv(GL.GL_VIEWPORT, viewport); //save viewport into buffer
gl.glGetDoublev(GL.GL_PROJECTION, projection); //save projection matrix into buffer
gl.glGetDoublev(GL.GL_PROJECTION, modelview); //save modelview matrix into buffer

viewport.rewind();
projection.rewind();
modelview.rewind();
objectPosition.rewind();

gl.glSelectBuffer(hits, selectionBuffer); //setup hits buffer

//int realY = viewport.get(3) - winHeight -1;
gl.glRenderMode(GL.GL_SELECT); //put gl context into selection mode
gl.glInitNames(); //initialise name stack
gl.glPushName(0); //push crap name onto stack to avoid error
//gl.glPopName(); //pop crap name off name stack

gl.glMatrixMode(GL.GL_PROJECTION);
gl.glPushMatrix();

gl.glLoadIdentity();
glu.gluPickMatrix(winWidth/2, winHeight/2, winWidth, winHeight, viewport);
//glu.gluPickMatrix(mouseX, viewport.get(3) - mouseY, pickWidth, pickHeight, viewport);
glu.gluPerspective(45.0f, ((float)winWidth/(float)winHeight), 1.0, 50.0);

gl.glMatrixMode(GL.GL_MODELVIEW);

//cube.drawCubes(gl);
c.draw(gl);

gl.glMatrixMode(GL.GL_PROJECTION);

gl.glPopMatrix();

gl.glMatrixMode(GL.GL_MODELVIEW);

gl.glFlush();
hits = gl.glRenderMode(GL.GL_RENDER);

//glu.gluUnProject(winWidth, realY, 10.0f, modelview, projection, viewport, objectPosition);

processHits(hits, selectionBuffer);

mouseLeftClicked = false;
System.out.println(gl.glGetString(gl.glGetError()));
System.out.println("oh we got to the end of picking; selectionBuffer has " + selectionBuffer.capacity() + " elements");
//System.out.println("gluUnProject results: x: " + objectPosition.get(0) + " y: " + objectPosition.get(1) + " z: " + objectPosition.get(2));
}
}

public void processHits(int hits, IntBuffer buffer)
{
cube.deselectAll();
System.out.println("in processHits; hits = " + hits);
if(hits > 0)
{
int aHit = buffer.get(3);
//gotta figure out how to select the cube with name buffer.get(3)
System.out.println("Your id: " + aHit);
}
}


Here is my current draw function:
public void draw(GL gl)
{
gl.glPushMatrix();
gl.glTranslatef(location.getX(), location.getY(), location.getZ());

if(amISelected)
{//have to put a glBegin and glEnd in here
gl.glColor4f(1.0f, 1.0f, 1.0f, 1.0f);
glut.glutWireCube(size + 0.1f);

System.out.println("You picked cube #" + name);
}

//ONLY WORKS WHEN IN GL_SELECT RENDER MODE
gl.glPushName(name);


gl.glEnableClientState(gl.GL_VERTEX_ARRAY);
gl.glEnableClientState(gl.GL_COLOR_ARRAY);
gl.glVertexPointer(3, gl.GL_FLOAT, 0, vertexBuffer);
gl.glColorPointer(4, gl.GL_FLOAT, 0, colorBuffer);

gl.glDrawElements(gl.GL_TRIANGLES, 36, gl.GL_UNSIGNED_INT, indicies);

gl.glDisableClientState(gl.GL_COLOR_ARRAY);
gl.glDisableClientState(gl.GL_VERTEX_ARRAY);

gl.glPopName();

gl.glPopMatrix();

}

Share this post


Link to post
Share on other sites
Not really sure what to tell you. You could find a simple implementation online that you know works for someone else and try using that as it is to see if you can get it to work. Then, try and figure out how they're different. I can't see anything immediately wrong with your code, but you never know.

Share this post


Link to post
Share on other sites
I figured out why it was returning -1. I had is as

gl.glSelectBuffer(hits, selectionBuffer); //setup hits buffer


hits is 0 here

but it needed to be:
gl.glSelectBuffer(BUFSIZE, selectionBuffer); //setup hits buffer


That said, it's acting wacky, not returning ids properly and sometimes not returning a hit when I don't click a cube straight on (in the case of the 1 cube)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this