Sign in to follow this  
resle

OpenGL [DX10] pixel-precise picking (reading Z values) [Yes, again!]

Recommended Posts

resle    129
Hi, I've been coming back and forth from the forums through the years, since the inception of DX10, which marked my final choice to move on from OpenGL. I've discussed the matters in several threads but still I couldn't get a straightforward technique about this... simply put: I am trying to pick a full x,y,z point from a rendered scene, using DX10. It's something that was fairly easy in OpenGL by unprojecting the x,y mouse pickpoint and then reading the final z value. Nonetheless, I am still failing to accomplish the same apparently simple task in DX10. Is there anyone perhaps that worked on this previously? Any hints? Thanks in advance andrea

Share this post


Link to post
Share on other sites
majek    115
There's a picking example included the DirectX SDK that uses D3D10, it uses the ID3DX10Mesh::Intersect function.

Share this post


Link to post
Share on other sites
resle    129
Quote:
Original post by majek
There's a picking example included the DirectX SDK that uses D3D10, it uses the ID3DX10Mesh::Intersect function.


Indeed... but that has nothing to do with my request :)

Share this post


Link to post
Share on other sites
MJP    19786
Quote:
Original post by resle
It's something that was fairly easy in OpenGL by unprojecting the x,y mouse pickpoint and then reading the final z value.


By that, do you mean sampling a value from the z-buffer?

Share this post


Link to post
Share on other sites
resle    129
Quote:
Original post by MJP
Quote:
Original post by resle
It's something that was fairly easy in OpenGL by unprojecting the x,y mouse pickpoint and then reading the final z value.


By that, do you mean sampling a value from the z-buffer?


Yes exactly: sampling a value from the z-buffer at given x,y coords, just after the whole frame has been rendered.

Share this post


Link to post
Share on other sites
Burnhard    96
http://www.gamedev.net/community/forums/topic.asp?topic_id=568139

There's a brief discussion of it here. I believe it's a lot more complex than it used to be with OpenGL :(.

Share this post


Link to post
Share on other sites
MJP    19786
I haven't done this myself, but I believe the best way to do this with D3D10 is to create a Texture2D resource with D3D10_USAGE_STAGING that has the same dimensions and format as your depth-stencil buffer. Then you use ID3D10Device::CopyResource to copy the data from your actual depth-stencil buffer to your staging resource, and you can then use ITexture2D::Map() to read the data.

You'll want to be careful though if performance is a concern, since doing a CopyResource to CPU memory will cause a stall to ensure that the GPU processes all pending commands before the data is copied. You can avoid this by double or triple-buffering your buffer so that you're not copying from a resource that is currently being used by the GPU.

Share this post


Link to post
Share on other sites
ET3D    810
It would be better to use CopySubresourceRegion to copy just the relevant pixel. Moves a lot less data to the PC side. However, copying depth this way is only supported on DX10.1 and higher.

Share this post


Link to post
Share on other sites
resle    129
Thanks Matt, thanks ET3D, and... yes Burnhard, it's a lot more complex than OpenGL's way. I wonder if doing that stalled GL drivers too or the architecture is completely different.

I will try with copying the... 1-pixel subregion I need, although I remember trying the whole staging/copy/subcopy one year or so ago without succeeding. Maybe I should try it under DX 10.1 rather than 10

Share this post


Link to post
Share on other sites
MJP    19786
Quote:
Original post by resle
Thanks Matt, thanks ET3D, and... yes Burnhard, it's a lot more complex than OpenGL's way. I wonder if doing that stalled GL drivers too or the architecture is completely different.

I will try with copying the... 1-pixel subregion I need, although I remember trying the whole staging/copy/subcopy one year or so ago without succeeding. Maybe I should try it under DX 10.1 rather than 10


I'd imagine that it did cause a stall...the problem is a consequence of the way the CPU and GPU work (asynchronously), and isn't caused by anything specific to D3D.

If you don't need to be very precise about your depth value, what you could do is shoot off a series of occlusion queries at increasing depths. That would only give you a range of possible depth values, but I'm not sure if that would be sufficient for your purposes.

Share this post


Link to post
Share on other sites
Darg    213
Assuming you plan to always have a decent frame rate then you could use the previous frames depth texture, I'd imagine that would cause less of a slowdown if you do it before starting your render loop for the current frame.

Share this post


Link to post
Share on other sites
Rompa    307
The other guys have covered things pretty well - may I ask *why* you need to pick an exact depth value?

Share this post


Link to post
Share on other sites
resle    129
Quote:
Original post by Rompa
The other guys have covered things pretty well - may I ask *why* you need to pick an exact depth value?


I need to attach some pixel-precise "pins" over geometry that's created procedurally by a geometry shader, so I don't have any cpu-held 3d mesh to perform intersections.

Share this post


Link to post
Share on other sites
MJP    19786
Is it possible then that you could position the pin entirely in the shader by sampling your depth buffer? That would simply things quite a bit for you, and would probably be the best performance scenario.

Share this post


Link to post
Share on other sites
resle    129
Quote:
Original post by MJP
Is it possible then that you could position the pin entirely in the shader by sampling your depth buffer? That would simply things quite a bit for you, and would probably be the best performance scenario.


It would be possible but extremely, extremely complicated too because of the general architecture of the rendering sequence in this software. Performance is not an issue anyway, the software in question wasn't even suppose to be "realtime" initially (biomedical stuff)

Share this post


Link to post
Share on other sites
resle    129
Quote:
Original post by ET3D
It would be better to use CopySubresourceRegion to copy just the relevant pixel. Moves a lot less data to the PC side. However, copying depth this way is only supported on DX10.1 and higher.


Hi ET3D, I hope you're "around" to see this :)

I did what follows with DX 10.0:
A) Created a 1x1 2D Texture with the same format of my depth/stencil buffer, and mae it a staging resource
B) Performed CopySubresourceRegion so that I copy exactly the pixel I need

I get no errors, and the same code works if use that to copy from, say, the backbuffer (despite the format is different!). Only, the returned pixel is always = 0

So I wonder what you mean that copying depth this way is only supported on DX10.1: by initializing the whole system for DX10.1 features rather than 10 would possibly fix things?
Thanks for the clarification(s),

andrea


edit: I noticed this just now

Quote:
CopyResourceSubRegion(...)
Differences between Direct3D 10 and Direct3D 10.1:

Direct3D 10 has the following limitation:
You may not use a depth-stencil resource as a destination.

Direct3D 10.1 has added support for the following:
You may use a depth-stencil buffer or a multisampled resource as a source or a destination.


The docs talk about depth-stencil as a destination anyway, when making a difference between 10.0 and 10.1

[Edited by - resle on May 19, 2010 4:18:40 AM]

Share this post


Link to post
Share on other sites
ET3D    810
Sorry I missed the PM. :)

I was basing my previous reply on that SDK comment, but reading it again, I think it would be possible to copy from a D24 to an R24 format.

Do you have debugging output enabled (D3D10_CREATE_DEVICE_DEBUG)? Is there any message printed for the copy?

Share this post


Link to post
Share on other sites
resle    129
Quote:
Original post by ET3D
Sorry I missed the PM. :)

I was basing my previous reply on that SDK comment, but reading it again, I think it would be possible to copy from a D24 to an R24 format.

Do you have debugging output enabled (D3D10_CREATE_DEVICE_DEBUG)? Is there any message printed for the copy?


Thanks a lot for coming back to the thread, your help has been invaluable to me in the past :)

IT's indeed possible to copy from D24 to R24 in general, I did that for other purposes. I've enabled DEBUG (and using the 10d.dll), but there's no error message.

The last thing I can do is stripping the whole zbuffer reading thing out of the engine and writing a simple app around it, something that does that and nothing else, and see if it works...

Just out of curiosity, as far as you know, does reading back the zbuffer stalls the pipeline in OpenGL too?

Share this post


Link to post
Share on other sites
Burnhard    96
It's hard to see how it wouldn't, given that both D3D and GL are layers on top of the same piece of hardware.

I'm back looking at this question again, except with D3D9. I'm veering towards a good old ray/mesh intersection test (optimised for the particular cases I have - i.e. a height-field). Depending on your data and any acceleration structure you're using, it might be the more flexible and overall the faster approach. Note that while your CPU is executing the code to do the intersection test, the graphics card can be busy rendering your scene - so you may win on performance there in any case, depending on the specific scenario you have.

Share this post


Link to post
Share on other sites
resle    129
Quote:
I'm veering towards a good old ray/mesh intersection test (optimised for the particular cases I have - i.e. a height-field). Depending on your data and any acceleration structure you're using, it might be the more flexible and overall the faster approach.


Only, I can't intersect cause I have no geometry data to use. It's a complicated scenario, and I don't even really have to "pick" something, I actually need the precise pixel in x,y,z space - that's it.

Anyway, in the loooong loooong loooong ago when Opengl 1.1 was the only choice out there, I've actually used that to perform geometry-independent picking. Think about it: you get the precise position of a point in x,y,z space. Good. Then you check this point againist the bounding box of an object, which may be a cube or a 10,000 triangles super complex shape. And that's all, you don't even have to transform geometry, interpolation or bone animation or whatever is automatically taken into account without further calculations.

Share this post


Link to post
Share on other sites
ET3D    810
When you get wrong answers the usual suspects are a driver bug or a bug in your app.

If you get the correct result when you use the reference device, it could be a driver bug. Many years ago (early D3D9 days) both NVIDIA and ATI had a bug where GetRenderTargetData didn't work correctly for small render targets (NVIDIA drivers could even freeze Windows in some cases). If it's a similar bug, you can do what I did then, which is report the bug and use a larger surface (such as 16x16) as a workaround.

If the reference device returns the same result, try checking PIX, to see where things go wrong. (You can probably just use PIX in the first place, if the reference device is too slow.)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this  

  • Similar Content

    • By Arulbabu Donbosco
      There are studios selling applications which is just copying any 3Dgraphic content and regenerating into another new window. especially for CAVE Virtual reality experience. so that the user opens REvite or CAD or any other 3D applications and opens a model. then when the user selects the rendered window the VR application copies the 3D model information from the OpenGL window. 
      I got the clue that the VR application replaces the windows opengl32.dll file. how this is possible ... how can we copy the 3d content from the current OpenGL window.
      anyone, please help me .. how to go further... to create an application like VR CAVE. 
       
      Thanks
    • By cebugdev
      hi all,

      i am trying to build an OpenGL 2D GUI system, (yeah yeah, i know i should not be re inventing the wheel, but this is for educational and some other purpose only),
      i have built GUI system before using 2D systems such as that of HTML/JS canvas, but in 2D system, i can directly match a mouse coordinates to the actual graphic coordinates with additional computation for screen size/ratio/scale ofcourse.
      now i want to port it to OpenGL, i know that to render a 2D object in OpenGL we specify coordiantes in Clip space or use the orthographic projection, now heres what i need help about.
      1. what is the right way of rendering the GUI? is it thru drawing in clip space or switching to ortho projection?
      2. from screen coordinates (top left is 0,0 nd bottom right is width height), how can i map the mouse coordinates to OpenGL 2D so that mouse events such as button click works? In consideration ofcourse to the current screen/size dimension.
      3. when let say if the screen size/dimension is different, how to handle this? in my previous javascript 2D engine using canvas, i just have my working coordinates and then just perform the bitblk or copying my working canvas to screen canvas and scale the mouse coordinates from there, in OpenGL how to work on a multiple screen sizes (more like an OpenGL ES question).
      lastly, if you guys know any books, resources, links or tutorials that handle or discuss this, i found one with marekknows opengl game engine website but its not free,
      Just let me know. Did not have any luck finding resource in google for writing our own OpenGL GUI framework.
      IF there are no any available online, just let me know, what things do i need to look into for OpenGL and i will study them one by one to make it work.
      thank you, and looking forward to positive replies.
    • By fllwr0491
      I have a few beginner questions about tesselation that I really have no clue.
      The opengl wiki doesn't seem to talk anything about the details.
       
      What is the relationship between TCS layout out and TES layout in?
      How does the tesselator know how control points are organized?
          e.g. If TES input requests triangles, but TCS can output N vertices.
             What happens in this case?
      In this article,
      http://www.informit.com/articles/article.aspx?p=2120983
      the isoline example TCS out=4, but TES in=isoline.
      And gl_TessCoord is only a single one.
      So which ones are the control points?
      How are tesselator building primitives?
    • By Orella
      I've been developing a 2D Engine using SFML + ImGui.
      Here you can see an image
      The editor is rendered using ImGui and the scene window is a sf::RenderTexture where I draw the GameObjects and then is converted to ImGui::Image to render it in the editor.
      Now I need to create a 3D Engine during this year in my Bachelor Degree but using SDL2 + ImGui and I want to recreate what I did with the 2D Engine. 
      I've managed to render the editor like I did in the 2D Engine using this example that comes with ImGui. 
      3D Editor preview
      But I don't know how to create an equivalent of sf::RenderTexture in SDL2, so I can draw the 3D scene there and convert it to ImGui::Image to show it in the editor.
      If you can provide code will be better. And if you want me to provide any specific code tell me.
      Thanks!
    • By Picpenguin
      Hi
      I'm new to learning OpenGL and still learning C. I'm using SDL2, glew, OpenGL 3.3, linmath and stb_image.
      I started following through learnopengl.com and got through it until I had to load models. The problem is, it uses Assimp for loading models. Assimp is C++ and uses things I don't want in my program (boost for example) and C support doesn't seem that good.
      Things like glVertexAttribPointer and shaders are still confusing to me, but I have to start somewhere right?
      I can't seem to find any good loading/rendering tutorials or source code that is simple to use and easy to understand.
      I have tried this for over a week by myself, searching for solutions but so far no luck. With tinyobjloader-c and project that uses it, FantasyGolfSimulator, I was able to actually load the model with plain color (always the same color no matter what I do) on screen and move it around, but cannot figure out how to use textures or use its multiple textures with it.
      I don't ask much: I just want to load models with textures in them, maybe have lights affect them (directional spotlight etc). Also, some models have multiple parts and multiple textures in them, how can I handle those?
      Are there solutions anywhere?
      Thank you for your time. Sorry if this is a bit confusing, English isn't my native language
  • Popular Now