My GJK implementation always terminates with a tetrahedron simplex if there is a collision, so that's not a problem. I almost have EPA working now. I added debugging code to draw the polytope and let me go through the algorithm step by step while rotating the view to inspect the polytope and make sure everything is working correctly. It expands the polytope correctly for several iterations, and then I start getting incorrect triangle normals. I'm almost positive I handle the triangle windings correctly, but I'll keep checking to find out what the problem is. As for the numerical stability... This is my first time writing a physics engine, so I don't expect everything to work 100% all the time
And I've never actually looked into using SAT in 3D. How is the performance? Does it work curved shapes? And does it compute the contact info?
Edit: I got it working now. It finds the correct penetration depth and normal. It seems to work very well. I've tested it with boxes and spheres of different sizes. I'll add other types of shapes soon. The only thing I'm confused about is how to find the contact points relative to each object