See, the traditional scene graph was a beast for many different purposes. Nowadays the recommendation in general is to use as many distinct (and decoupled, if possible) structures as needed, where the particular structure should be task oriented instead of being made matching a omni-potent scene graph. There're many ways to go, of course. The following is just one:
IMHO frustum culling is a task similar to zone containment, collision detection, proximity calculations, and similar things. They all compute the spatial distance of a more or less simple volume against a couple of other simple volumes. Think of a SpatialServices sub-system that allows to send appropriate queries to it. When, for example, rendering is about to be invoked, compute the frustum volume from the current camera set-up and query the SpatialServices for the "list of all object IDs that collide with the given volume". Notice that although frustum culling as a task is still existent as usual, its main work is externalized to a more general sub-system, and the sub-system is not aware of what frustum culling is.
Of course, it may happen that unforeseen requirements occur in a later development phase. Either the then existing services are suitable to fulfill them, or else you need to either extend / re-factor them or create new services. If a second (i.e. different) partitioning scheme is required, then ... so it is.
Also, if I'm understanding this correctly, there's no real need to store game actors through a special spatial partitioning, rather a simple std::map<ID, GameActor*> would suffice?
You need a kind of database where all existing game actors (or more general, entities) are stored in. A map would be sufficient for this, that's correct (letting aside any eventual performance issues). This structure allows you to retrieve an entity by its ID, and it can be iterated to touch all entities. That is its point. It doesn't mean that the entity cannot be found in a spatial partitioning, too. But it should be stored there as a bounding volume with an ID, and not as a game actor or some other high level thing.