I suggest you keep to using a 2d physics library. You should be able to shape the world and the object definitions so they do what you want. It's going to make your life much easier.
The statements in the above post are short-sighted., especially about collisions. What happens when 2 objects collide, but when you handles the collision responses, one of the objects collide with a 3rd object that's already been checked for collisions? And, you'll have to properly handle all types of angles and surfaces for these collisions. Physics engines handles these for you, and are VERY efficient at doing it.
Thank you BeerNutts. The diffrerent contact vary from different position. I define 4 states to decide which force to apply. So I can control it now.