Managing engine subsystem interactions

Started by
5 comments, last by Ashaman73 8 years, 11 months ago

This question was kind of derived from the other thread,

http://www.gamedev.net/topic/668960-engine-design-global-interfaces/

But since it has degenerated into another singleton war, I guess I'll start a new one.

I'm currently using the "Passing around pointers to anyone interested." method. But since I have a huge set of subsystems, and I don't know who could be interested in what, I wrap all the subsystems as forward declared pointers in a giant context object.

Suppose a feature is one kind of entity will play a footstep sound according to the material it is standing on, syncing with its' animation, this is what I'll do:


class Foo :public Entity{
  //called every frame,
  //or maybe a event triggered by animation frame or physics contact...
  virtual void update(){
    if (this.animation.isFootstepFrame()){
      //find what we are standing on
      //don't have to be a raycast, could be contact checking, sensors, etc...
      Entity ground = this.context.physics.Raycast(this, downward);
      
      //Entity have properties as string-string pair
      string materialName = ground.getProperty("Material");
      //lookup the actual material from its' name
      Material material = this.context.MaterialRegistry.getByName(materialName);
      string stepSound = material.getStepSound();
      this.context.audio.playSoundEffect(stepSound,this.position);

      //do other stuffs
      //a lot of subsystem is involved
      this.context.camera.shake();
      this.context.scene.AddEntity(new FootPrint(this.context, this.position);
      /*
      FootPrint::FootPrint(...):
        Entity(context)
      {
        this.texture = context.resourceManager.getTexture("footprint.png");
        ...
      }
      */
      this.footStep.RaiseEvent(this);
      /*
      somewhere else
      void OnFooFootStep(Foo *f){
        f.context.logger.log("footstep");
      }
      */
    }
    //the script could do a huge variety of stuffs,
    //possiblely using any subsystems.
    this.script.run(this);
  }
};

This method is working great for me, for now.

I read that context objects are considered anti-pattern, but don't understand why. Also with all those "contexts" floating around and easily retrievable, I feel it is not a lot better than a global, other than I can have multiple contexts.

Are there better solutions than this "I don't know what you'll need so I'll give you everything"?

Advertisement

Well, it is difficult. A context has many advantages which can help your code a lot, on the other hand, it has many disadvantages which could destroy your code. Eventually is it a matter of choosing the right dose.

A context can help to manage cyclic dependencies, on the other hand it can lead in too many, intransparent dependencies.

A context helps to avoid a lot of refactoring, because you already have a way to access new functionality, on the other hand it hurts when refactoring, because you don't know exactly where something uses a certain state of the context.

My personal opinion:

A context is useful (e.g. the lua state in Lua is just a context) and is not evil per se, but you should avoid overusing it. Just adding the context to every object is a bad idea (=global state), but as pointer passing around to methods which are in need of the context, and not just for being potentially useful in the future, makes it more transparent. This way you can detect method, which will use the context in some way. An other way it to subdivide your context to more logically parts, e.g. a render context, audio context, script engine context etc., if you use a global context which contains the sub contexts, try to use the specialised context whenever possible over the global context.

Example:

Not the best option:


MyGameObject {
   Context mGlobalContext;

   void playSound(soundname) 
   {
      mGlobalContext.soundContext.playSound(soundname);
   }
   void otherFunction(xyz) 
   {
    ...
   }
}

Better:


MyGameObject {
   

   void playSound(Context& globalContext, soundname) 
   {
      globalContext.soundContext.playSound(soundname);
   }
   void otherFunction(xyz) 
   {
    ...
   }
}

Personal favorit


MyGameObject {
   

   void playSound(SoundContext& soundContext, soundname) 
   {
      soundContext.playSound(soundname);
   }
   void otherFunction(xyz) 
   {
    ...
   }
}

I read that context objects are considered anti-pattern, but don't understand why.
...
I feel it is not a lot better than a global

Seems you do know why then :)
While better than a global, it's still got a foot in the realm of dependency-sphagetti, where you never know which memory regions will be read/written when and by whom. The flow of control and the flow of data are... meandering.

If the flow of control is sticks with each logical system for longer blocks of time (instead of jumping between different objects/tasks constantly) and larger batches are operated on, then not only will single-core performance increase, but it becomes much easier to split work over multiple cores, and IMHO, the data-flow oriented nature of such designs makes your engine easiet to understand /maintain to boot!

Suppose a feature is one kind of entity will play a footstep sound according to the material it is standing on, syncing with its' animation.
this is what I'll do

To get practical with the above advice -

virtual void Entity::Update() hints that your main loop has no knowledge of the flow of data within a frame. It's knowledge of the frame probably consists of: "for each thing, do the stuff".

Instead of this vague view of what happens in a frame, imagine (or draw) a directed-acyclic-graph of all the function calls that occur in this main loop. Color code them by which 'system' they're in (audio/raycast/etc). Then draw out boxes representing all the variables/state in the game, and draw arrows between the graph nodes (function calls) and the data boxes, with arrowheads imdicating reads/writes.

It looks like spaghetti doesnt it? :D

The objective is to make that diagram clean enough to actually have a chance in hell at drawing it on paper. To do that, you need a better main loop than "for each thing, do the stuff".

The first thing your Foo Update does, is read some data from an animation object. This is a temporal data dependency - the animation object must be updated before the Foo object! Easiest way to ensure that is for the game loop to update a list of all animatiojs before updating a list of all Foos.

Drawing the flow between animations::update and foo::update is simple now. Instead of constantly jumping back and forth, there's a big block of animations followed by a big block of foo's. More importantly, the main game loop now knows that it's operating on animations and foos, not abstract entities, so it can pass the specifically required objects into their update functions instead of a bloated context object.

Foo's update also does nothing if an animation event hasnt occurred, which seems likely, so with a lot of foo objects, this is a lot of wasted polling.

Consider using events instead of polling here. When the FooEntity is created, it can register itself with it's animation object, to be notified when a footstep happens. While you're at it, why not use an ID to represent "footstep" instead of hardcoding that feature (data-driven makes it flexible).

Now, when the animstion event (a footstep) occurs, don't go repsonding to the event bu calling Foo::OnAnimationEven right away - that breaks up your batch processing. Instead, have the "for each anim, update anim" loop (maybe called an animation system) produce a list of events that occurred. The Animation::Update function can take just this list as an argument, and push events into it.

After updating all animations, you can sort the list of events by recipiant-type (e.g. some events will go to Foo::OnFootstep, some will go to Bar::OnJump). By sorting the list, you can batch process all Foo anim events back to back, instead of jumping back and forth between Foo logic and Bar logic.

The next thing you do is a raycast. This is typically an expensive operation as it has to read a large amount of world collision data from RAM into CPU cache... So again, we want to batch up as many raycast calls as possible and then execute them all back to back.
Again, build up a queue of raycast requests - Foo::OnFootstep can push a request into this list, along with a function to call if the ray hits something. That function can take the entity and material as parameters.

Later in the frame, you have two loops:
"for each raycast request, cast ray, if hit, record results"
"sort raycast results by callback type"
"foreach raycast result, call callback"

You can then build a list of footstep locations and pass that list onto two functions -- one that spawns new footstep decals, and one that enqueues new footstep sounds.
...etc...
That's one basic pattern anyway... Break up your game frame loop into the actual flow of data that's occurring, and thrn manually schedule the different systems within that data flow, giving each the minimal inputs and outputs.

If you can create a nice data flow directed-acyclic-graph, its super simple to schedule parallel parts of that graph across CPU cores, and get simple multithreading without much extrs thought too.


Seems you do know why then smile.png

I know globals are bad only because people say so, but I didn't fully understand why they are bad, especially the public access part. With your explanation now I know how they are bad for cache coherency.

I still have some questions regarding your engine structure.

Different features might have data the flow through the subsystems in different order, and might go through the same subsystem multiple times. For example, we might need to do several consecutive raycasts that depends on each others result. How do we ensure all the works are done before we finish a single frame? Do we start by raising a update event and loop through all the subsystems until there are no more works queued for this frame?

By batching jobs subsystems will operate on a command pattern. I once used a engine whose rendering system is done like this, but debugging it was quite a headache. With procedural methods, if we received a segfault or other exceptions, all the inputs and callers down until main() is probably still on the stack so it is easier to track where it went wrong. But with a command pattern the information of who gave us this garbage is lost, unless we do some excessive logging. For the footstep example, suppose one of the many sound effects played is wrong, how would you suggest to track down where went wrong?

Thanks a lot!


I know globals are bad only because people say so, but I didn't fully understand why they are bad, especially the public access part. With your explanation now I know how they are bad for cache coherency.


I still have some questions regarding your engine structure.

Different features might have data the flow through the subsystems in different order, and might go through the same subsystem multiple times. For example, we might need to do several consecutive raycasts that depends on each others result. How do we ensure all the works are done before we finish a single frame? Do we start by raising a update event and loop through all the subsystems until there are no more works queued for this frame?

By batching jobs subsystems will operate on a command pattern. I once used a engine whose rendering system is done like this, but debugging it was quite a headache. With procedural methods, if we received a segfault or other exceptions, all the inputs and callers down until main() is probably still on the stack so it is easier to track where it went wrong. But with a command pattern the information of who gave us this garbage is lost, unless we do some excessive logging. For the footstep example, suppose one of the many sound effects played is wrong, how would you suggest to track down where went wrong?

Thanks a lot!

Many programmers are quite dogmatic. They get disproportionally upset about things, often when they don't even really know why they are supposed to be bad.

For example a static class member is 'global' in some sense but really it avoids 99% of the danger of global variables. If you are older and you dealt with old code, you know why real global variables CAN BE REALLY REALLY BAD. I still see it sometimes for various reasons but anyone who went to some basic classes or even a programming forum heard how bad it is. I don't think it's something to really worry about today, especially a global static class member.

I guess an entity class kind of spells out why it's hard (maybe pointless) to try and limit out what accesses what. You really need to be able to do 'whatever' from your entity class. But your application is driven by what you do, not the other way around. If that's what drives it, that's what drives it. But you don't necessarily even have to have an entity class or custom scripts. If you can identify what you actually will do for all cases, you can simply give those properties to the right 'manager' code in your predefined order and let it go to work.

Remember this is just an optimization though. You probably really don't want thousands of entities all playing their own sounds anyway! If you have just 5-10 characters on the screen at once, really who cares. Are you making a game or trying to seel someone an engine?


This is my thread. There are many threads like it, but this one is mine.

So in this case instead of calling on the entity to do stuff each frame, you send the entity around to various interested parties, partially process it and move on.

I think the biggest mistake people make in design is typically they shoe in data and code together. Oh I have this type of data, better make a data processor for it. Code objects go with your functionality, not your data. Once you figure out your process well then the proper code structure will follow from that.

This is my thread. There are many threads like it, but this one is mine.


By batching jobs subsystems will operate on a command pattern. I once used a engine whose rendering system is done like this, but debugging it was quite a headache. With procedural methods, if we received a segfault or other exceptions, all the inputs and callers down until main() is probably still on the stack so it is easier to track where it went wrong. But with a command pattern the information of who gave us this garbage is lost, unless we do some excessive logging. For the footstep example, suppose one of the many sound effects played is wrong, how would you suggest to track down where went wrong?

Regardless of having a context or not, a command buffer/queue/message system etc. which would result in a debugging headache, is a good way to construct an asynchronously procssing framework, which on the other hand is a very good foundation for a distributed system. Modern games are more or less distrubuted system nowadways with different components and hardware running in many threads. This will be more or less the game-engine future. (Btw, you can enchance such a system with debug information to track down bugs).

Coming back to context, in distributed systems a context is always difficult, because it needs synchronization and a consistent state.In a game a context is therefor sub-optimal, but atleast we have a consistent state most of the time, still you need find a way to access it in a performing way (eg lockless, read-only etc.).

Context are sometimes necessary (ever tried to implement a transaction in a real distributed system without context wink.png ), often handy, and always alluring. In your example you use a context as global variable with lot of synchronized calls to different subsystems. If you ever want to utilize multiple cores, then you will run into issues.

This topic is closed to new replies.

Advertisement