Dear sirs,
I have a (hopefully very silly) question about a collision detection problem that I'm unable to solve myself

I need to check collisions between a (well, some) tri-mesh based moving object and a very simple fixed 3d "dungeon". The "dungeon" has a flat floor (a simple plane, trivial to check collisions there) and some tri-mesh walls around (walls can be very long and complex, however their triangles are mostly oriented in vertical). There are plenty of examples about reducing the complexity of collision checks (through trees and so) and I've already cut the problem down to just checking some (few) triangles of my 3d object against some (few) triangle candidates of the wall.

Again, checking triangle-triangle collisions is quite trivial (there are plenty of examples and algorithms around), but my problem is that my physics engine needs compenetration to calculate collision forces so I need to know not only where collision points are (easy), but also "how deep" the collision is and the collision "normals" in order to apply collision forces that "bounce" the object off the wall.

Now, this is a bit of a problem because if I cut down the problem to just a triangle-triangle collision it becomes difficult to get the right amount of compenetration and, most important of all, the right colliding "normal"; as an example, in a triangle that belongs to a perfectly flat wall surface I can get (depending on how the triangles of the object mesh are compenetrating the wall) a triangle side-to-triangle side collision, with a collision "normal" pointing in an "impossible" physical direction. In such a collision the only "right" normal would be the wall triangle normal, because the triangle is part of a wider flat surface and its sides cannot "generate" collision points, but how can I avoid this in the last, triangle-triangle collision detection step?

I've tried to investigate the gjk method but this only works for convex shapes and my walls are too complex to be reduced to convex pieces (they may be long and very twisty, with a complex local surface), so a triangle-triangle based algorithm would be preferable; anyone has any idea on how to solve this problem?

Thank you very much for any help!

I can't quite envision the situation where this happens. There is probably a fundamental flaw with how you're doing it. Anyway, if it is indeed happening perhaps you could check for potentially "bad" collisions, e.g. very minimal contact area or odd angles of contact, and if so use the triangle's neighbouring triangles to create a pseudo-plane to get a normal from? I say pseudo-plane because I doubt all points will be co-planar.

I can't quite envision the situation where this happens

Well, look at the picture attached. The grey triangles are part of the wall, and the red one is part of the object. The red triangle is colliding with the wall (the 'c' vertex is penetrating the wall with a penetretation depth showed by the green arrows). Problem is that the red triangle is in collision with two wall triangles, both sharing the a-b side, and when the narrow-phase triangle-to-triangle collider calculates the penetration depth it will report a side-to-side collision between one (well, both) of the grey (wall) triangles a-b side and the red (object) triangle c-d side (showed by the blue arrows), because maybe the "blue" penetration depth is smaller than the "green" one. So, the collision normal will be reported along the blue line, whereas it's quite obvious that this is a "wrong" collision normal and that in that case the only "right" collision normal should be along the green line. Any subsequent penalty force applied to the red object, of course, will be wrong and the object will rebounce in a "wrong" way (which is, btw, exactly what is happening to me...)

Also consider separate collision geometry. Graphic meshes build up in complexity much faster than collision can handle.

Yes, you are right, but from my previous example you can see that I'm having problems even with a very very simple geometry (a perfectly flat wall made by a simple triangle strip). Thank you, however, for the gimpact link, I'll give it a look!

Well, look at the picture attached. The grey triangles are part of the wall, and the red one is part of the object. The red triangle is colliding with the wall (the 'c' vertex is penetrating the wall with a penetretation depth showed by the green arrows). Problem is that the red triangle is in collision with two wall triangles, both sharing the a-b side, and when the narrow-phase triangle-to-triangle collider calculates the penetration depth it will report a side-to-side collision between one (well, both) of the grey (wall) triangles a-b side and the red (object) triangle c-d side (showed by the blue arrows), because maybe the "blue" penetration depth is smaller than the "green" one. So, the collision normal will be reported along the blue line, whereas it's quite obvious that this is a "wrong" collision normal and that in that case the only "right" collision normal should be along the green line. Any subsequent penalty force applied to the red object, of course, will be wrong and the object will rebounce in a "wrong" way (which is, btw, exactly what is happening to me...)

I see. It seems to me that you need to make some assumption to resolve this. For example, if you assume that all meshes are closed, it is impossible for a triangle to collide with the edge of another triangle. Therefore you can either assume the collision is in the direction that the object is moving, or that it's in the direction of the normal of the triangle, depending upon which assumption works out better. If you draw a line from point c in the direction of either of the grey triangles' normals until it hits the plane of either grey triangle, it will be roughly the green line.

Looks like the best thing to do here is to use bounding spheres. It is easy to calculate bounce with spheres. This should help
http://www.gamasutra.com/view/feature/3015/pool_hall_lessons_fast_accurate_.php