For collision detection my engine implements a mix between GJK and the Xenocollide algorithm with Minkowski Portal Refinement (MPR) to get contact points. You can find the algorithm described in detail in Game Programming Gems 7 section 2.5. A simple 2D example is given on the authors website. Other special-case detection I follow Real-Time Collision Detection by Ericson modifying to my specific needs.
For resolution I used a mixture of the following resources to understand the equations of motion and implement a sequential impulse solver:
- Game Physics Engine Development by Ian Millington-- while a good resource for general physics engine development, I do not recommend following this approach if you want a stable system for stacking more than 3 or 4 boxes high. The latest edition has an addendum for 2D physics if that is your interest.
- Game Physics by David H. Eberly -- This book is very thorough and gives lots of details on the math for resolution. At an implementation level this book tends to be more focused on block solvers and simultaneous resolution.
Finally, I followed Box2D lite for ordering the integration / detection / resolution pipeline as well as for how to resolve via impulses only (with no direct position corrections), and how to best do warm-starting.
All this said, the two features that added the most to the stability were:
1) Warm starting on normal and tangential impulses
2) Impulses for penetration correction
3) Adjusting the coefficient of restitution (bouncy objects don't lend well to stacking). Box2D lite effectively sets this to 0 along the normal if you look in the code.
Hope this helps!