The result of all this is that I'll call an Update() somewhere, which ends calling functions across multiple systems without any upper-level coordination

The only other "way" though that I can think of is deferred execution. In some cases this works pretty well; I will frequently use a CollisionManager that will detect collisions between objects, but won't actually make any immediate changes. Instead, it produces records which are later processed by the objects themselves to make changes to state. This is great, but it is effective primarily because it only entails one extra pass, in which all handling can be performed.[/quote]This is a much better style -- find some operation (e.g. collision handling) and cut it out into it's own isolated unit - the decreased coupling produces better, faster, more predictable, smaller code.
Since the PS3 came out, console developers have almost been forced to working in this way, as the SPU's can't handle code that randomly calls functions or access random memory -- each SPU only has 256KiB of RAM (which is often reduced to 128KiB to allow streaming of jobs), and all of the code/data that you want to operate on at one time has to fit in that tiny area. This really forces you to break up processes into a flow of buffers and data transforms.

For example, object A casts a "spell" on object B, which causes objects C1, C2, and C3 to be created. And on that same frame, the whole lot of them need to simultaneously enter a coreographed animation ( this is a real example! ). In order to accomplish this without object A directly calling functions on B and C1-3, there would need to be 3 rounds of message-passing to be handled, with a ton of parameters being passed around. Theoretically possible, but painful. Yet, perhaps I just haven't found the right framework for this sort of system? I'd be open to working this way, if it can be proven to be flexible and involve minimal overhead.

The above collision system example should be what you strive for in all systems -- taking a large input buffer (e.g. shapes+positions) and producing a large output buffer (e.g. collision results). I'm sure you could decompose the above into a "new effect" message buffer, and a "apply spell damage" message buffer, etc, etc...

There are more general purpose approaches to deferred execution though, for example check out the Actor Model, which is basically OOP, but every method call is written to a queue before being executed at the appropriate time. I've implemented this model in a game engine before, and it can be done very efficiently.

In my current engine, I'm half-way through implementing by general-purpose message buffering system, which works as below:
A Lua script can write code that looks and behaves like typical code:b:TakeSpellDamage(a, 42)
anim = new.AnimationTimeline()
c1 = new.Effect(anim)
c2 = new.Effect(anim)
anim.Start(0, 42);

But the actual C++ code that's being called under the hood looks like:TaskQueue& q = ...;
Push_B_TakeSpellDamage(q, *b, a, 42);//copies a function pointer, the argument arguments and 'this' into a queue, which can be run later or concurrently in another thread.
Future anim = Push_AnimationTimeline_New(q);
Future c1 = Push_Effect_New(q, anim);
Future c2 = Push_Effect_New(q, anim);
Push_AnimationTimeline_Start(q, anim, 0, 42);

//later...
q.Execute();//actually execute the above callsThe crazy code to bind C++ classes to this system looks something along the lines of:class AnimationTimeline //example class
{
public:
AnimationTimeline();
void Start(float beignTime, float endTime);
};
//bind the class to the scripting system
eiMethod2( void, AnimationTimeline, Start, float,beginTime, float,endTime );
//etc...


#define eiMethod_( R, T, Name, C, Args, List, Store, Get ) \

struct T##_##Name##_Args { Store }; \
void Call_##T##_##Name(void* user, void* blob) \
{ \
T##_##Name##_Args n = {}; \
T##_##Name##_Args& a = blob ? *((T##_##Name##_Args*)blob) : n; \
((T*)user)->Name(Get); \
} \
void Push_##T##_##Name(TaskQueue& q, T& o C Args) \
{ \
T##_##Name##_Args a = { List }; \
q.Push( &Call_##T##_##Name, &o, &a, sizeof(T##_##Name##_Args) ) \
} //

#define eiMethod0( R, T, Name ) \
eiMethod_( R, T, Name, , , , , ); //

#define eiMethod1( R, T, Name, A0,N0 ) \
eiMethod_( R, T, Name, COMMA, A0 N0, N0, A0 N0;, a.N0 ) //

#define eiMethod2( R, T, Name, A0,N0, A1,N1 ) \
eiMethod_( R, T, Name, COMMA, \
/*Args */ A0 N0 COMMA A1 N1, \
/*List */ N0 COMMA N1, \
/*Store*/ A0 N0; A1 N1;, \
/*Get */ a.N0 COMMA a.N1) //