Unprojecting: 1000 questions, 0 answers
Author
I've spent a whole evening on google looking for an adaptation of the good old couple GlReadPixels / GluUnproject that using OpenGL allowed with relative ease to know which [wx,wy,wz] world point your [sx,sy] mouse pointer was on.
I've been reading tons of examples about intersecting, raycasting, basic usage of D3DXVec3Unproject but nothing that, in the end, obtains the same streamlined goal (mostly, if I understood correctly, because there's no proper way to read a single ZBuffer value (!!)).
So in the end, is this somehow possible using DX, even at the cost of dirty tricks?
Thanks
a.
Author
Further into the subject..
the reason for trying to get at all costs the exact world projection of a screen x,y coord is, of course, picking.
I already understood how to construct a "ray" using D3DXVec3Unproject, and eventually intersect it with objects. So for each visible object I'd do:
1) Test if the ray intersects with the object's bounding box (fast)
2) If so, test if the ray intersects with any of the object's triangles (slow!)
Now think about this: if you can directly obtain the wx,wy,wz world coord you're pointing with your sx,sy mouse pointer, all you must do is check if that 3d point falls into a bounding box. If so, you don't have to check intersections with every single object's face, because you can already be sure that the point is "touching" the object's surface.
(this method worked flawlessly and super fast with opengl)
the reason for trying to get at all costs the exact world projection of a screen x,y coord is, of course, picking.
I already understood how to construct a "ray" using D3DXVec3Unproject, and eventually intersect it with objects. So for each visible object I'd do:
1) Test if the ray intersects with the object's bounding box (fast)
2) If so, test if the ray intersects with any of the object's triangles (slow!)
Now think about this: if you can directly obtain the wx,wy,wz world coord you're pointing with your sx,sy mouse pointer, all you must do is check if that 3d point falls into a bounding box. If so, you don't have to check intersections with every single object's face, because you can already be sure that the point is "touching" the object's surface.
(this method worked flawlessly and super fast with opengl)
Hey. Let me answer another question instead.
Easiest way to do picking is often this:
* Render object-ID/triangle-ID in 16x16 pixels around the pixel where the mouse cursor is. You can do frustum culling vs this small rectangle so most objects are rejected.
* Read back to CPU and check. Fairly fast for small framebuffers.
Hard part is precision, but if you set up your ID's correctly you can obtain back the ID correctly on the CPU.
If you don't have precision to have a unique ID for each triangle in the scene, consider:
A) first determine object-ID and then polygon-ID within that object using two passes.
B) write two integers (perhaps using multiple render targets). Object-ID in the first one and polygon-ID in the second one.
Why is this easier? You can reuse your renderingversion of your object. No need for collision friendly versions at all. If you want to know where in the triangle you are picking, you have to do ray<>triangle-intersection, but that is easier than doing ray<>scene-intersection.
I guess you could do the same thing to get a small zbuffer using shaders and storing depth encoded as colors somehow (just as you store int as colors in the approach outlined above), but then you still have to find where that point is in your scene (ie on what triangle).
Easiest way to do picking is often this:
* Render object-ID/triangle-ID in 16x16 pixels around the pixel where the mouse cursor is. You can do frustum culling vs this small rectangle so most objects are rejected.
* Read back to CPU and check. Fairly fast for small framebuffers.
Hard part is precision, but if you set up your ID's correctly you can obtain back the ID correctly on the CPU.
If you don't have precision to have a unique ID for each triangle in the scene, consider:
A) first determine object-ID and then polygon-ID within that object using two passes.
B) write two integers (perhaps using multiple render targets). Object-ID in the first one and polygon-ID in the second one.
Why is this easier? You can reuse your renderingversion of your object. No need for collision friendly versions at all. If you want to know where in the triangle you are picking, you have to do ray<>triangle-intersection, but that is easier than doing ray<>scene-intersection.
I guess you could do the same thing to get a small zbuffer using shaders and storing depth encoded as colors somehow (just as you store int as colors in the approach outlined above), but then you still have to find where that point is in your scene (ie on what triangle).
Author
Sorry but.. the method you describe is beyond my current DX skills or comprehension of the pipeline :
All my picking happens when the scene has been fully rendered. At a pure D3D level, I don't need to know what triangle or even just what object I am picking. I'd just need to fully unproject a 2 coord to 3d.. done that I already have a working picking/collision framework. The little step I miss is that damned unprojected point..
All my picking happens when the scene has been fully rendered. At a pure D3D level, I don't need to know what triangle or even just what object I am picking. I'd just need to fully unproject a 2 coord to 3d.. done that I already have a working picking/collision framework. The little step I miss is that damned unprojected point..
Quote:Original post by resle
...
Now think about this: if you can directly obtain the wx,wy,wz world coord you're pointing with your sx,sy mouse pointer, all you must do is check if that 3d point falls into a bounding box. If so, you don't have to check intersections with every single object's face, because you can already be sure that the point is "touching" the object's surface.
(this method worked flawlessly and super fast with opengl)
i doubt your method is flawless even in opengl. how about if two objects' bounding boxes share the point at the same time? which one will you choose?
if you are using ray-triangle intersection test, the ray equation is probably like this: P + t * D. in the test, you are probably interested in the t scalar value, just pick the least positive value.
generally, you do not want to test each triangle in an object, you will want to build some kind of spatial partitioning structures for the triangles, e.g octree, k-d tree, and have it filter some candidates for you. i have written octree, k-d tree, i also used opcode, now i am using bullet's btBvhTriangleMeshShape. the usage difference between opcode and bullet's is that in opcode you only need to provide the point and the direciton of the ray, but in bullet's you need to provide the starting point and the furthest point of the ray.
of course, these only apply for static geometry. for dynamic geometry, you will want to use some approximating bounding volume.
so the following shows the flow of my triangle selector:
set min_t to infinityfor each object if the ray doesn't intersect with the object's bounding volume, skip to next object transform the ray with the inverse world transformation of the object so that we obtain a local ray use the spatial partitioning structures of the object to filter some candidates for each triangle if the ray-triangle intersect fails then skip to next triangle if t is larger than or equal to min_t skip to next triangle set min_t to t store some information, e.g. object pointer, triangle indexif min_t is not infinity, something is picked, otherwise nothing
Author
@hellknows2008:
Yes my method wasn't 100% flawless but for my purposes it worked just great. I've never experienced an overlapping issue as far as I can remember.
That's the approach I need to reproduce, because of 2 factors:
1) My objects are animated
this changes their geometry in time, so checking rays againist the
transformed vertices is another problem I wouldn't like how to solve.
(I am using hardware accelerated tweening)
2) Speed speed speed
I am checking what's under the mouse cursor not less than a thousand times
for rendered frame.
So I wonder once again... is it possible that isn't there any way to simply unproject screen coords??
Yes my method wasn't 100% flawless but for my purposes it worked just great. I've never experienced an overlapping issue as far as I can remember.
That's the approach I need to reproduce, because of 2 factors:
1) My objects are animated
this changes their geometry in time, so checking rays againist the
transformed vertices is another problem I wouldn't like how to solve.
(I am using hardware accelerated tweening)
2) Speed speed speed
I am checking what's under the mouse cursor not less than a thousand times
for rendered frame.
So I wonder once again... is it possible that isn't there any way to simply unproject screen coords??
I'm assuming you're using Direct3D 9 (Direct3D 10 has a lot more features which can ease picking). The standard way of obtaining Z information in D3D9 is to render it yourself, by passing depth values from the vertex shader and writing them into a floating point render target. You can take a look at the ShadowMap sample in the SDK to see how that's done.
Author
@ET3D : Yes I am using Dx9, I should have specified that, sorry. (Just out of curiosity... what makes picking easier in Dx10?)
As of now, I was trying to lock a 1x1 rectangle out of the DepthStencil buffer using LockRetangle.. but looks like there's no way of initializing with the LOCKABLE buffer format despite I have a a last generation card. Bah!
So you suggest to pass values from the vertex shader.. it would be a good idea if the overhad isn't going to be insane (for example being able to pass just the Z information relative to the single (x,y) depth buffer element I am interested into!) I will look up the example you suggested.
As of now, I was trying to lock a 1x1 rectangle out of the DepthStencil buffer using LockRetangle.. but looks like there's no way of initializing with the LOCKABLE buffer format despite I have a a last generation card. Bah!
So you suggest to pass values from the vertex shader.. it would be a good idea if the overhad isn't going to be insane (for example being able to pass just the Z information relative to the single (x,y) depth buffer element I am interested into!) I will look up the example you suggested.
Quote:Original post by resleSo I wonder once again... is it possible that isn't there any way to simply unproject screen coords??
You cant directly read the Z-buffer in DX9...
Therefore, if you want to use a z-value after rendering you must explicitly render this z-value into a screen-sized floating-point texture, or cast a ray.
If you don't want to do any collision, then you must render the depth value into a texture, then do unproject using mousex, mousey, and z-value from the texture...this should work.
However, I don't think this would help you with selecting of objects.. what good is a 3D value anyway? you still need to do collision detection of some kind.
A better way would be to render every object into a buffer with a special unique ID number assigned to each one...then do a check against the texture using the screen coords.
Author
@Matt
This sounds feasible. Let me see if I got it: I should redraw the whole scene or at least part of it using some shader that stores Z-values into a texture as large as my backbuffer.. then I read values from the texture. Right? Any code hints?
I don't know how to explain it in an easily understandable way, mainly because english is not my mother tongue.. Well, if you know exactly what 3d point you're "aiming", all you need is a check of that point against a bounding box. If the check returns that the point is within a given object's box, you can be sure that you've picked that object with *pixel* precision, without other checks! Not only: this method also takes into account stuff that's drawn using an alpha channel. Imagine a cube textured with many "holes" due to its texture's alpha. Geometry-based pickers will return a positive check even if you click in the hole!
Hmm, what I seem to fail to communicate is that I don't need to strictly pick an object. Think of another scenario: you've got a 3d terrain working with LOD and other tricks that add/remove vertices and faces dynamically. Then you need to click somewhere on that terrain and have some object move there: how do you get the point where the object should go? You can't use intersections, most of the gometry has been generated either by a shader or something else. Using intersections, too, would return - at best - the clicked *triangle*, certainly not the exact world point.
Quote:If you don't want to do any collision, then you must render the depth value into a texture, then do unproject using mousex, mousey, and z-value from the texture...this should work.
This sounds feasible. Let me see if I got it: I should redraw the whole scene or at least part of it using some shader that stores Z-values into a texture as large as my backbuffer.. then I read values from the texture. Right? Any code hints?
Quote:However, I don't think this would help you with selecting of objects.. what good is a 3D value anyway? you still need to do collision detection of some kind.
I don't know how to explain it in an easily understandable way, mainly because english is not my mother tongue.. Well, if you know exactly what 3d point you're "aiming", all you need is a check of that point against a bounding box. If the check returns that the point is within a given object's box, you can be sure that you've picked that object with *pixel* precision, without other checks! Not only: this method also takes into account stuff that's drawn using an alpha channel. Imagine a cube textured with many "holes" due to its texture's alpha. Geometry-based pickers will return a positive check even if you click in the hole!
Quote:A better way would be to render every object into a buffer with a special unique ID number assigned to each one...then do a check against the texture using the screen coords.
Hmm, what I seem to fail to communicate is that I don't need to strictly pick an object. Think of another scenario: you've got a 3d terrain working with LOD and other tricks that add/remove vertices and faces dynamically. Then you need to click somewhere on that terrain and have some object move there: how do you get the point where the object should go? You can't use intersections, most of the gometry has been generated either by a shader or something else. Using intersections, too, would return - at best - the clicked *triangle*, certainly not the exact world point.
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
