That’s the right idea. Basically the way I deal with this problem in my game is for each unit to keep a list of ids of other units in the flock; whenever two units collide, they offset their destinations so that they’re moving in parallel to areas around the ultimate destination (if they’re moving in the same general direction) and add their own id to the lists in all these connected units so they can keep track of each other. Flocking and keeping a list also means you can eliminate a lot of unnecessary collision detection every update loop, since you know that units that have already been adjusted to move as a group won’t collide with each other again unless some outside unit interferes.
There are a lot of special cases to check for, but having some kind of list of nearby units means you can adjust the acceptable distance to the target depending on how many units are around. If you’ve just got one unit, it has to move exactly to the target, but if there are a lot of units moving to the same place, you can have a larger radius based on the number of units and their individual sizes for defining whether each one is close enough for the move order to be satisfied. (They should still keep moving to their destination at least until they collide with another unit, but once they do, they’ll check if they’re already within the group-defined radius of the destination and then either stop or tell the units blocking the way to move.)