I finally managed to put together complete system that will be used as a backbone of my new game. The game itself is supposed to be a mix of Kerbal Space Program and Space Engineers in 2d, maybe with some added influences from other games. I'll get back to this later, when gameplay is more fleshed out. Right now what the game needs is: orbital physics of planetary system, large deformable terrains with rigid body simulation and transitions between those two.
tl;dr; There is a video.
This part was relatively easy. I choose simulated n-body system over fixed orbits with patched conic approximation. It's simple to write and supports all phenomena like Lagrange points. My system uses Verlet integration with fixed time step (64s) for planets and adaptive time step for small entities (artificial satellites, asteroids). The only drawback here is that plotting trajectories requires to know positions of all planets ahead of time, so it's best to evaluate whole system many steps to the future and just playback current position from history. (You can see these in video when debug drawing is enabled)
I modeled planet's surface as linear space located around planet radius, with wrapping around when traveling sideways. Since I want my planets big (not real live big, not even KSP big, but still), each one is split into spaces that span around 4km.
Those spaces are all linear and centered around origin. Rigid bodies can move from one to another, can have joints across spaces, but will never collide or interact in any way.
Of course the problem is when some body wants to leave one space and enter another. We could simply teleport whole rigid body, but it would miss some collisions.
My solution is to clone whole body at new location and link them with teleport joint. It's similar to fixed joint, but maintains relative offset. (don't forget to split mass in half between two bodies)
In order for this to work, you need two bounding rectangles: tight and fat. When tight intersects space boundary clone body (2). Destroy cloned body only if fat stops intersecting boundary (4).
For all this to work I made some changes to Box2D library, you can find my fork on github.
Stitching it together
Moving from planet surface space to orbital space is using simple teleportation with conversion from local space coordinates to global orbital one. I sill have some quirks to iron out, but hey, you can fly to the moon!