Sign in to follow this  
Prune

Multisampled buffer screws up color-based picking at edges even when MSAA disabled!

Recommended Posts

After SDL_SwapBuffers I render the objects for picking as follows: // Lighting is always disabled //glScissor(event.motion.x - 1, SH - event.motion.y - 1, 4, 4); glDisable(GL_MULTISAMPLE); glDisable(GL_BLEND); glShadeModel(GL_FLAT); glEnable(GL_DEPTH_TEST); glClearColor(0.0f, 0.0f, 0.0f, 0.0f); glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT); // Render stuff unsigned int pixel(0); glReadPixels(...); // Figure out selection by value in pixel Now, I'd expect the edges to be sharp, either picking color or background. Instead there is still some blending--even if I disable MSAA in the driver control panel! This completely ruins the picking since I want the ability to slide the cursor over objects, and thus edges need to be handled properly. Indeed, it's not an issue limited to polygon edges: one of the objects uses a texture with colored areas as a map on the object for selection. Even though the texture uses GL_NEAREST for minifying/magnifying, the edges still are not completely unfiltered (even if I turn off anisotropic filtering in the driver). Picking near edges does work fine if I comment out these initialization lines: SDL_GL_SetAttribute(SDL_GL_MULTISAMPLEBUFFERS, 1); SDL_GL_SetAttribute(SDL_GL_MULTISAMPLESAMPLES, 4); However, then I can't get AA on the visible rendering pass. So, what do I do? The AA is important in this project for the visible pass, but accurate picking is also important as it triggers menu items. The thing is, I have _many_ items drawn for selection so I can't just use very distinct colors and reject non-recognized ones--plus, that's not very robust. And I can't just direct the user to only click in inside objects, as I need the ability to slide the mouse from one object over another and they get selected in turn. I don't know what's causing the problem and I need help.

Share this post


Link to post
Share on other sites
A more robust and efficient method of picking, especially for menus and other common 2d controls, is to use geometric primitives such as rectangles or polygons and testing whether the pick point is inside them or not. This also eliminates all and any dependence on the graphics system regarding the pick algorithm.

If you need to pick 3d geometry, ray-triangle intersection is also robust and not as expensive as you might think - especially if you can use simplified meshes optimized for picking.

Share this post


Link to post
Share on other sites
I'm aware of the ray-intersection approach, but first, that won't allow me to pick from texture colors--i.e. imagemap on a 3D object, and second, the size of this project makes a rewrite of the selection system out of the question. I'm looking for a fix, not a different approach. This far my idea is rendering the picking to a GL_RENDERBUFFER_EXT, which could be only 2x2 anyway.

Share this post


Link to post
Share on other sites
It is entirely feasible to derive texture coordinates from a ray-triangle intersection by solving the barycentric position of the intersection point in the triangle space, interpolating the vertices' texture coordinates with the barycentric vector, and using the resulting texture coordinate to sample a given texture.

That said, I do see the point in not wanting to rewrite the logic; however, due to anti-aliasing being not entirely in your control, approach suggested by me might be the only way to get guaranteed exact results not depending on graphics drivers and/or hardware.

Share this post


Link to post
Share on other sites
Quote:

The thing is, I have _many_ items drawn for selection so I can't just use very distinct colors and reject non-recognized ones--plus, that's not very robust. And I can't just direct the user to only click in inside objects, as I need the ability to slide the mouse from one object over another and they get selected in turn. I don't know what's causing the problem and I need help.


I post the following reluctantly (probably because I don't have a solution for you but just some info on a point of interest).

Firstly, how many is _many_ items?
500, 1000, 10,000?

As far as using distinct colors for selection: It can be a very effective and consistent method of selection (for its level of simplicity). Even though I must agree that the ray-intersection approach mentioned is more robust.
e.g. I have a psuedo-random color generator that assigns a unique color to each model and (sub-model) mesh that is loaded. This color is applied to the model/mesh when rendered in wire mode (at model level or mesh level) or when selection is performed. i.e. each model/mesh is just rendered with its flat color material (i.e. in it's unique color).
So far, I can have a maximum of 768 unique colors available for this purpose(i.e. I have 128 discreet colors (of unsigned byte/char rgb form), where the r,g,b values are swapped in all there possible permutations (3!) to produce the 768 possible. It does take a bit of care to ensure you do produce colors distinct enough for the models to be uniquely selectable, but it's pretty trivial(tedious?). When needed, I will be extending this to say 256 colors with the 3! permutations to produce say 1536 unique colors?. obviously there will be/are limitations to this scheme( as schemes go). And if I wasn't such a lazy bum, I could even work the mathematical combinatronics for discreet unsigned bytes (for the 3(rgb) values 0 to 255)that make unique colors (where there needs to be workable distinction of color values for selection work as expected: Gee I hope that makes sense!)

However, after all that, none of this color discussion stuff above will be relevant to you if that multisampling, blending, AA problem persists reguardless.

Just one more thing: with my little uniquely assigned coloring/selection scheme I know I have unique colors to select from. If you are using various colors and even textures. How are you ensuring 2 objects don't have the same texture ( i.e. pixel color)? I am thinking maybe you have one and only one "object that uses a texture with colored areas as a map on the object for selection" or something?

Anyhow, as I said, this post is more about a point of interest that an actual solution to your problem, especially since I have no idea about the impact of the multisampling, blending, AA problem and the coloring scheme I mention.

Share this post


Link to post
Share on other sites
Texture colors could be from a reserved range. In my case, I match them to object colors where I specifically need clicking on said objects to trigger identical effects as when clicking on the texture.

Share this post


Link to post
Share on other sites
Instead of rendering to the default surfaces that get created with your OpenGL context you could try to render to a Framebuffer Object with a Texture or Renderbuffer attached as target. This should solve your AA problems since a normal FBO can't do MSAA since it does not normally allocate a multisample buffer. Rendering to a normal texture will certainly fully disable MSAA since normal textures don't have multisample buffers.

Share this post


Link to post
Share on other sites
[b]Warning: This thread is from 2008.[/b]

Thanks Trenki!

I was trying to turn on FSAA for visible rendering, but it would mess up my color picking mode. Your suggestion worked for me.

Also, thanks to forum search for pointing me to this thread, so I didn't have to create a new one. :)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this