Casey from Molly Rocket Games gives an excellent presentation on GJK here.

He goes through the 2 point and 3 point cases very thoroughly and his code for the 4 point case is online here.

I've implemented this method and had no problems with precision using 32 bit floats. In order to avoid the infinite loop problem, I just clamped the maximum number of iterations to something like 20 which is plenty. If the limit is reached then there is no collision.

Casey doesn't show GJK, but only a boolean operation whether two objects intersect. GJK is an algorithm to find the closest points between *disjoint* convex polyhedra. I think Gino called this GJK-SAT in his book (s. also Christers post). Still some of the ideas like the simplex solving can be used with GJK as well. The termination condition doesn't work though.

Some of the numerical problems with GJK can be solved by solving in object instead of Minkowski space, but then it is hard to detect penetration. Also when using GJK/EPA you need a third algorithm to jump in when EPA fails (what can actually happen when you don't terminate with a 4D simplex). So I agree it is a mess and since you only find one contact point per frame you need to use an incremental manifold and will suffer from tunneling issues where objects can rotate out of the world at contact if you don't work around this. Personally I also prefer SAT, but GJK/EPA seems faster, but much less robust. So it is a question what quality you expect. Personally I think we should do less, but more higher quality physics in games.