10 replies to this topic

[BytePtr]

Hello.

In my editor im drawing selectionboxes with OpenGL, they are drawn using glDrawArrays from pre calculated vertex coords.
From const array.
The box(boxes) are textured from texture loaded from file (PNG) and processed with SDL to make texture for OpenGL out of it.


The drawing and stuff works fine but there is very annoying problem (bug) for me that is really really ugly for me.

Look at the GIF.




As you see, first there are just white rectangles on red/black texture and if mouse cursor hit's the WHITE border of selection boxes, then boxes jump around (up and down).


Is it possible to tell OpenGL to ignore the selection boxes texture or the WHITE BORDER "width" in pixels or something like that?
To finally stop this jumping once and for all?

glReadPixels maybe could help, but if the textures have the same color? It will jump again?

I must tell you that it always should ignore selection boxes, but if there is anything else under the mouse cursor it shouldn't ignore it and then it is allowed to jump around.

It just must ignore the white selection boxes. That's all.


Any help, ideas, tips or hints are very very welcome.

[szecs]

Um.... I don't understand the problem. It seems to me that the problem has nothing to do with openGL and rendering, but the logic.

Maybe you could show the whole code or you should explain what you want to achieve (the final goal, not the logic to achieve it).

[Tachikoma]

How are you actually testing for selection anyway? I mean if you test the point against a rect, you are either inside, or outside, that's it. Edge cases are special and generally require extra checking.

[BytePtr]

szecs

Well, look at this gif then.




There is a mouse dragging and red selection rectangles. In this gif you see that if mouse cursor goes over the red lines, then the selection boxes are not going up like in my first gif. They go up only if there is something else under cursor (not selection boxes itself), in this case there is a little box at the right, as you see.

In first GIF, when the mouse cursor goes over the white line, the boxes will "jump up".
But they shouldn't, it must ignore the white lines as the second GIF shows, the red lines are ignored under mouse cursor.


Basically it same as telling: selection shouldn't select itself!

Um.... I don't understand the problem. It seems to me that the problem has nothing to do with openGL and rendering, but the logic.

Maybe you could show the whole code or you should explain what you want to achieve (the final goal, not the logic to achieve it).

[BytePtr]

How are you actually testing for selection anyway? I mean if you test the point against a rect, you are either inside, or outside, that's it. Edge cases are special and generally require extra checking.

Basically im not, most of the work is done by OpenGL with gluUnProject.
And im drawing the boxes at the coordinates, which gluUnProject returned me.


Same code as here:
http://nehe.gamedev....nproject/16013/


I think that's the glReadPixels that finds the textures. Because i think that if there is empty "world" and i click in this world, it reads: 0,0,0.
So there is nothing under cursor.

In my case it finds anything that's not 0,0,0 and his friend gluUnproject tells me the coordinates where there is something under cursor.
And also it finds the selection boxes texture and that could be reason for jumping around.

[Tachikoma]

The images only show the symptom. You still haven't explained how your algorithm works.

Are you testing cursor intersection against a geometric object? Are you testing against rendered objects in a buffer?

edit: Never mind, just read your second response.

I think you should cast a ray through the cursor and explicitly test boxes intersecting against it. So instead of using buffers to perform the test, you are doing it in the geometric domain.

[Brother Bob]

I also have a bit difficulty understanding the problem; but perhaps it's because the solutions is too easy... because if

Basically it same as telling: selection shouldn't select itself!

then the solution is to not draw the selection geometry when selecting. Selection is only performed on what you draw when selection is enabled, so if you don't want the selection to select itself, then you don't draw the selection.

[JTippetts]

AFAICT, the linked NeHe code uses the depth value to perform the reverse projection, so you could try disabling depth write via glDepthMask(GL_FALSE) when you draw the selection box so that the box wireframes don't pollute the depth buffer.

[BytePtr]

AFAICT, the linked NeHe code uses the depth value to perform the reverse projection, so you could try disabling depth write via glDepthMask(GL_FALSE) when you draw the selection box so that the box wireframes don't pollute the depth buffer.

Wow, this is excellent. It really works.

Basically i did this:

glDepthMask(false);
DrawSelector;
glDepthMask(true);

Looks so smooth and much nicer now.
Can you explain a bit more why it helps and works?
Or could post some link where i could read about it? I want to learn also from it.


Thanks alot for you and the others here who replied.

You guys got your +

[JTippetts]

The Z-buffer is used by the renderer to determine which fragments to keep and which to discard, based on their Z-depth, or distance from the camera. A side-effect of this is that when rendering is done, the Z-buffer contains a depth "map" of the scene. Since any given point on-screen reverse-projects to a line-segment in world space bounded by the near and far planes, the only way to determine the exact point in world space that a screen point reverse-projects to is to read the Z-depth from the buffer and use that as the Z component of the coordinate to reverse-project. This is what the NeHe example is doing by calling glReadPixels to assign the value of winZ.

Now, when you draw your selection boxes with depth-writing enabled, then the Z value in the depth buffer that is used to reverse-project with will be equal to the depth of the wireframe of the selection box at that location, rather than the depth of the level geometry behind it that you are trying to select. This apparently fools whatever code you use to calculate the selection boxes into thinking the level geometry depth is higher than it really is. In effect, the algorithm doesn't discriminate between the "real" stuff (level geometry) and the "fake" stuff (the selection box). By disabling depth-write when you draw the selection boxes, you are making those boxes a visual effect only; they have no presence in the depth buffer, and thus don't over-write the depth values from the level geometry. In effect, they now "don't count" for the purposes of reverse-projecting.

[BytePtr]

Excellent explanation, thank you very much.
Another + for you.