Try this:
Each unit in the formation has a prefered target position relative to the lead unit... e.g. "back 5 over 3" For each unit in a formation, you calculate a pathfinding routine to move toward that relative position. If the leader has to squeeze through an opening only wide enough for one, the followers will follow the wall to the door and file through one at a time in their quest to re-establish formation. Once on the other side of the opening, they would then move (hopefully unobstructed) toward their positions.
It''s all about telling the units where they need to be... which just happens to be a moving point behind the leader.
Dave Mark
Intrinsic Algorithm Development
