For various reasons, I need to hold tiles, objects and enemies in seperate managers - but what if I had a manager for all physical objects (that contained pointers to said objects) - did a quick broad phase test, then proceeded to run a more fine grain AABB check?
Sure, sounds reasonable. That's pretty much I'm doing with the GameState class, except there's no broad phase collision testing yet, and I don't plan to add it until I start seeing performance issues with many many many objects in play.
I say, stick with one manager. It would probably make sense to keep different kinds things (tiles, enemies, etc.) in separate lists/arrays, but this is kind of a trivial detail. Let the manager handle it's own internal state. Just do whatever works and keep it simple.