Right now I am looking at using a system similar to oliii's cube3d demo, using SAT to find overlaps. I would need to change a couple of things in order to support spheres & ellipsoids with that system, but it shouldn't be too hard.
This morning I have started adjusting how A* is used by the AI. First I made it so that the A* will only take a certain # of maximum steps per Think(). This prevents the fps from diving when the players get somewhere the AI can't follow.
Next I need to change the way the A* is invoked, and it brings up some interesting gameplay choices.
We have a top-down view, and it's possible at times to see objects & enemies from this view that would otherwise not be visible from the player's perspective. This is generally a good thing, b/c it helps the player more effectively play tactically, and fight multiple enemies.
But, should the AI have the same ability to find the player?
One school of thought is that this is one of the player advantages over the AI, and that the AI should remember the last time it saw or thinks it heard the player, and use that as a general direction or destination for its path finding, but not omnisciently know where the player is at any given time.
Another way of looking at it is that the AI should be able to 'see' a player that is on screen from its POV. This would be a bit easier code-wise, but possibly too hard for the player.
We plan to have stealth in the game, so I'm leaning towards the first approach. It would add more suspense if the player were hiding in the shadows, hoping not to be seen by an enemy just around a corner.