I want to be sure I understood your solution.
- this is meant to avoid the redundant checks on edges and verts.
- each leaf <=> triangle 'owns' only some of what's normally its edges and vertices so that for the whole mesh, one vert or one edge belongs to one and only one triangle.
Then 6 bits thing is a far more elegant solution than my previous fixed size array solution. And you add only 1 byte per face which is pretty small. Sounds very good. Do you have some money for a patent ? ;)
The only minor problem is that this bitfield must be passed to the triangle/triangle col. det. primitive. Not a big deal. You can check the other thread about tri-tri. I think Voronoi can be exploited to make it more speedy than the brute separating axis technique used by Oii (already efficient though). In the case of the separating axis, it will be easy to add something like if(!bDisabled[iAxis]).
Globally, with a well optimized algo for the sphere-tree/sphere-tree part, the whole should give a very efficient technique for predictive collision detection of arbitrary meshes.
Here is an idea to cull much of the tree/tree recursion in case of time based col. det. At each step of the recursion, the predicted time intervals of potential contact should be processed chronologically. And also the biggest spheres should be visited in priority.
This way :
-----------------------------------------------------> time4) X[--- removed ----------------------------] ([SS1.1/SS2.1])3) [SS1.1/SS2.2] ([---------------SS1.2/SS2]-----])2) [---------SS1.1/SS2-------]1) [-------------------SS1/SS2--------------------]
(SS1.1 is one of the sons of SS1, etc ...)
X is a potential contact detected.
So the recursion should target to find two leaves has soon as possible (example SS1.1/SS2.2). The goal is to find an exact potential contact between triangles (that will actually happen if no other contact is found earlier). Next the time of this contact (tmax) will reduce the time interval of search. This will cull most time intervals later, very high in the recursion hierarchy.
Once tmax has been found it's probably not useful to compute at the triangle/triangle level during the recursion. Unless the leaf/leaf interval has an upper boundary lower than tmax. Then there is a chance to make tmax even lower, which will help culling even more.