I'm trying to implement RTS style units movement, where you can click on a point, and all units move into there in a group.

Right I have it almost functional, the units move in a group, and don't intersect each other.

However it seems that the correct method is not stable enough, units will constantly move and bounce around even after reaching the targeting point.

My current algoritm is the following:

1 - Calculate direction vector to target

2 - For each unit find the nearest unit

3 - if they intersect, move this unit a very small bit in the opposite direction

I tried doing the collision test with all other units instead of only the nearest, but the result is similar (all units pushing each other around endlessy). What am I missing?

I guess this problem is somehow similar to a very simple 2D engine, so the solution is probably related to that, but my maths is quite weak