Designing a "message based" engine...

Started by
17 comments, last by GameDev.net 18 years, 1 month ago
Oh wow, funny I just saw this thread. The past week, I've been experimenting with a different architecture that was "message based" similar to what you are talking about. I'm not going to give away eveything yet, because I'm still working with the idea and I like it so far, so here's some commentary on what you have said in addtion to what I've come across:

Quote:What I mean by a "message based" engine is that parts of the program submit "messages" into a queue. ... Each frame the engine (if you can call it that) goes through the queue and looks at each message. Depending on the nature of the message, the engine makes calls to methods in other parts of the program.


The first time I started this project, I actually tried the idea of keeping a message queue, then dispatching the messages by allowing each object to know what message to start at, so it can iterate though and process the messages. The main problem with this approach is first the overhead of storing messages in your queue. When do you remove them? What if a message needs to go to several different classes?

From what I saw, storing all of the messages then passing them to the objects could also create desynchronization. For example, if I passed two messages, one side effect would be that one object would "catch" up by processing the events in the message queue until there were no more and then the next object would do the same. What resulted was some objects doing messages out of order, which was bad. Now I could have made it so objects only processed messages one at a time, but then I was thinking, what's the point of that?

Next I saw if I was just going to be passing messages as they occured, why store them? If they only need to go to a specific object, then message processing could just send it as it is received, thus eliminating any desyncing as well as the additional overhead. With that in mind, I then based the rest of the design in terms of an "instant feedback" approach. Now, as you may be wonder, and as wa asked, "what about delayed events". At first I was thinking that was the only thing my approach could not accommodate for, but then I thought about it.

A delayed event is an event that is sent at some time, to which it actually happens a later time. More specifically in real life, it's some 'cause' that triggers some 'effect' in the future. Now, with the old version of storing messages in a queue, you could allow that system to track the time and dispatch messages only when some delay reaches 0, but I did not think that was realistic. My whole idea fo this "delayed event" is that an event is sent to some object, and the object itself handles the delaying.

For example, if you set an alarm clock to go off at some time, the object itself controls when the "effect" actually happens. If that object was turned off, then the event would not happen at all, which is more realistic with my model than that of the conventioanl message queue, which would still dispatch that message to the object. In addition, the actual messaging system would have to be some centralized component that received messages anddispatched them while keeping track of time and who it can send to and all of that. I opted for something much different.

Quote:I thought about creating a generic base class for a message. It would contain a few simple variables, such as a name, a boolean value to see if it has been taken care of and can be discarded, and perhaps a few other basic things like that. Now I would derive other classes from this, depending on what the message needs to contain.


Good start with the idea of a generic base class for a message, that is what I do currently. However, I left it at that. Simple and easy to use, there is only one message class that stores 3 things, who sent it, data of the message, as well as the names of the message. The reason I did this was to minimize the dependecies that would otherwise be created with your proposal. For example, if you have 3 different classes with 3 different messages, all 3 would have to know about each other's format of the message if they want to interact.

I took the approach of "void* message passing". Yes, this of course seems more dangerous and unsafe, but, it's one of those things that is fine to do when you know exactly what you are trying to do. In my system, I pass a 1:1 ratio of named messages with void* data. This way, any object that receives a message can parse the message name and if it implements it, then it can process the data accordingly. My approach does rely on more of a "you know the data being passed around", so it is more error prone, but with a reasonable amount of checking, it is very flexible.

Quote:The problem I am having is, how would the engine be able to distinguish between the different types of messages? How would the engine 'know' that a message contains variables specific to that kind of message?


As mentioned in the previous paragraph, the message name will let the object know what the message is. "What is done" with the message is specifically up to the object itself. If it does not handle the named message, then it will naturally just ignore that message. If it does handle it though, then it can process the data if any. Since each data will have a name with it, the end user can handle data in any order it comes. What I was going after with this approach is that of more a life-like representation. For example, let's say I pass a "kick" message to a boulder. Naturally, the boulder will not handle that message since it will not be affected by it.

Now I just said if I pass a "kick" message to a boulder, it would not handle it. With my design, all messages are sent to all objects! So if the player "kicks", all messages will receive the message as well as the data associted with it, such as where it took place, how hard was it and so on. If an object does not handle the kick message, it will simply ignore it. If it does handle it though, then it can see if it is actually affected by it.

Next comes the whole aspect of knowing what messages to pass and all of that. At first I just typed in the messages using strings and handled the handling of them doing the same. The problem that I saw was if that I needed to change a message one place, I'd have to change it in the other place, so what I did was make one header file that defined all of the messages being used. Of course each time I made a new message, I'd have to recompile, but it was a lot clearned and easier to use.

With my current design, I will be finding another way to do this, something that involves a map to which I can just register events and use them without having to worry about inconsistances. The idea would be that I can register messages and track them, so at any time, if a message is used that is not registered I can stop execution and warn myself that a typo was made or something was used that needs to actually be handling. That will make debugging a lot easier as well.

One last response on the idea of the actual messages being used. My main goal is to define a set of engine specific messages that are used to denote common events, such as when a key is pressed, what key it is will be passed as well. By doing this, I can then tie in any framework or library to use my system which would make integrating it a lot easier. In that case, all you would have to do is add in the specific message broad casting of events that corerlate to the engines defined ones and you are all set.

Ok, now I have covered pretty much the basics of my system. Time for demos! As I went along developing this system, I needed a way to test it in application to see where any problems were. What resulted was me making a SDL Selection Box demo. I have screenshots of the demos, the demos themselves, as well as screenshots of code to give you the idea of what is going on.

After I got to version 8 of that last example, I felt it was time to move on, so I hacked together an example, but this time using 3D and Opengl . OpenGL Selection Box. In that example, I implemented my messaging architecture and saw a few things had to be changed. Ok for controls:
Left mouse - Select or draw drag rectangle
Arrow Keys - When an object/objects are selected, change their rotation
Pg Up/Down - When object(s) are selected, will change their Z values.

That's about it for that demo, the hardest part for me was actually getting the OpenGL stuff to work correctly (one little thing I was doing wrong in my picking that threw me off for hours).

That's about it for now, I am working on integrating this system into a new engine that will use it. By doing this, I can see where some problems might occur or additional functionality I might need. For example, I always had the ability to send a message to a specific object, based on it's guid, but with the logics I had, it was being done incorrectly. Final note on this system are that it is decentralized -- there is no main message component. As to what I do, you should be able to figure it out though the code screenshots.

If you have any questions on this (Yes I see that I kinda wrote a lot [grin]), feel free to ask! I'm not giving away any code right now though, but I can show certain specific parts. On a final note, you can take aa look at my old threads at attempts to this as well: Void* Messaging for Classes, Self Managing Objects - Part 2, Self Tracking Object Design

Phew, that took a while [smile] Good luck!
Advertisement
Wow, thanks Drew! I think it will take a while for my poor brain to absorb all of this!

For anyone else, feel free to add your comments if you have any!
@Drew:

On Timers:
I agree that it would be simpler for objects to handle the "delay" and then fire an event when the timer comes for it. That's what my engine does. Letting the engine handle time sorting of events is an extra, unneeded hassle. Letting objects handle their own problems frees the engine up and makes it run cleaner IMHO.

On Queues:
I agree on getting rid of queues as well. It seems to make more sense just to just say:

event.DoIt();

than to say:

Engine.YouDoItInstead( event );

@Moe

On Maps:

Maps are your friend in this case. Instead of doing this business:

if (eventname == EVENT_MOUSECLICK) { do stuph }
else if (eventname == EVENT_KEYPRESS) { do stuph }
... etc ...

... you'll want a map. This assumes you have 1.2 bajillion message types and it's getting messy. With a map, you just do like so:

message_map["mouse_click"].DoIt();

On Problems:

The biggest problem that you'll find is that events, or what handles events, needs to know about other objects. Let's say for instance that the key "ENTER" is pressed on the keyboard. This shoots a gun in your game. Well, the problem is that the engine or event must know about this said gun and must also know how to operate it. The naive way would be like this:

Engine::HandleKeypress( key ) {   if ( key == "ENTER" ) {      gun_pointer->Shoot();      }   }


But as you can see, the engine needs a pointer to the gun. You will find as you grow that the number of objects and things your messaging system needs to be aware of is going to get ugly. You'll need pointers to the player, the gun, some timed events, GUI widgets of various flavors, the network if multiplayer, and all sorts of other silly little things. This is really ugly.

Ideally you need a system where the "engine" has no concept of how to specifically handle events. All it does is process them. You can do this in a few ways. None of them are ideal.

One way is the Command pattern, which i have used with some success. A command is an "event" that is linked to a user-input action of some kind.

Here is a basic class for a command:
class Command {public:	Command()  {};		virtual ~Command()  {};	virtual void Execute() = 0;	};

And here is an inherited Command that lets the player shoot:
class CommandPlayerFire: public Command {public: 	CommandPlayerFire( player* p ) {};	 	~CommandPlayerFire() {}; 	void Execute();	private:        player* p; 	};


So what you do is create a Command and attach it to something. All the "something" needs to know how to do is the Execute() function:
// create a shooting command:Command* shoot_command = new CommandPlayerFire( pointer_to_player );// "register" it with the engine-thingy somehowengine->RegisterCommandWithKeypress(KEY_ENTER, shoot_command):


Then somewhere deep in the belly of the event processing system, you have this:
Engine::ProcessEvent( event ) {   if ( event.type == KEYPRESS ) {      key_map[KEY_ENTER]->Execute();      }   }


Then the engine doesn't even have to know what kind of Command is attached to the ENTER key. All it knows is that when ENTER is pressed it needs to Execute() the command attached to it.

Gotchas:

This sort of thing works pretty well for certain situations but you may find problems:

Let's say the player dies:

// player death:
delete player;

Oh! Now press ENTER!

// press enter
*** segmentation fault ***

The pointer to the player is now invalid.

Which means that your objects and your commands have to have some sort of communication if this sort of thing happens.

If you do this, you also need a way to update/add/remove commands registered with your engine.

Anyway, i hope that gives you some ideas. I'm not saying this is the silver bullet, it's just another approach to think about.
I am also using a message based system within my engine but since my engine uses the systems approach the messaging is just a small part that allows the subsystems to talk to each other without knowing if they are there. In C# the delegates/events really help this since anyone can listen to an event that gets fired when a new message is sent through the top system. My message class has a few things...

Source - Where the message came from.
Destination - The intended destination of the message.
Title - Usually a command of sorts.
Message - The actual message.
InnerMessage - A recursive link to the same Message class just in case I need it. This is of course optional.

One thing I have really liked about the message system is I have a console class that takes input from the user at runtime and sends a message if the user types the send message command. Since all objects in the game's systems are added through the root manager I keep a reference to them and using Reflection I am able to change there properties pretty easily at runtime.

So messaging of some sort, I would say, is a nice feature if not a must in most engines today. I am currently in the process of adding the property changing feature, but if you want to take a look at my code and whatnot just click the link below. I am providing lots of documentation as I go along.
Oh, another alternative to "messaging" is a signal/slot system. It's pretty slick.

clicky 1
clicky 2
clicky 3
Quote:Original post by leiavoia
Which means that your objects and your commands have to have some sort of communication if this sort of thing happens.


In Game Coding Complete 2, Mike McShaffry addressed this problem quite well. Everything in the game had a unique ID; so your actors would Have their own Ids, so would sounds, textures, etc. Instead of passing a pointer around in the messaging system the game would pass one of these Ids around; when you try and resolve the Id to a pointer (dereferencing the handle) it will return NULL if the object has since been deleted, thus adding in a safety mechanism to the system.
I just got a response back from cow_in_the_well. I figure it is relevant, so I will post it here:

Quote:Original e-mail from cow_in_the_well

In MGE I toyed around with some messaging systems mostly as an abstraction between Python and C++, but it never got used substantially. A win32 style MsgProc type messaging thing is the simplest way to go about things, but I prefer to go about things in more of a C++ manner.

The first step towards this would be to use the base Event class system that you mentioned. The base event class would store an event ID, and a derived class would store the data for that event ID. The problem I have with this method is that your message handler, which would take an Event* would have to upcast the class type from Event* to the specific event type in order to get the data, but upcasting is technically "unsafe" without using dynamic_cast (C++'s built in RTTI), but dynamic_cast isn't fantastic either. Upcasting is what I did in MGE, e.g:

if (event.GetType() == KeyPress)       KeyPressEvent* kpe = (KeyPressEvent*)event;

A way around this would be to have the Event class handle the event itself, via a virtual function in the base class such as Event->HandleEvent(), but this sort of defeats the purpose of messages as the message itself is now acting as the message handler (although the event class could simply be a proxy that calls some other handler; for example the KeyPressEvent could look up a key bindings map that maps a key to a specific game play function pointer).

Really, the biggest issue is where and who handles the actual message. The simplest way would be to have a single monolithic win32 style MsgProc that gets events and then dispatches the event to whoever needs to know, but this leads to too much potential of having high level gameplay logic in the low level part of the engine (unless you post off events to another handler until it propagates up to the higher levels).

The way I did it is to have a base EventListener class, and instances of this class could be registered with the singleton EventManager class. When an event gets triggered, all the event listeners (who asked to be told about that type of event) will get their virtual HandleEvent function called. This allows for a more dynamic handling system where event listeners can be plugged in and out at runtime.

In any case, for my current version of MGE (and for Stick Soldiers for that matter) I am not using a pure message based system for the entire engine. It's mostly just a traditional game loop that goes through and updates each subsystem one after another.

For the places where I need some kind of abstraction from whoever uses the event (such as key input), my primary method at the moment is to use a quake 3 style keybinding system that binds keypresses to console commands. This provides abstraction at pretty much the highest level (code that has access to the engine core can, however, get key states directly from the engine if the want).

For example, I can go into the console and type: "bind W forward".

This simply associates the "W" button on the keyboard with the console command "forward", so that "forward" is called once per frame while the button is held down. It's important to note, however, that when you bind something to a button it is simply associating a raw string with the button (so could type "bind W aergnaerugi" and it would be bound, but it would not do anything when pressed since aergnaerugi is not a valid command -_^.

This being the case, we can do funky stuff like

bind F10 "toggle fullscreen; restartvideo"

which is actually running two console statements. "toggle fullscreen" toggles the fullscreen console variable from true to false, and then it runs "restartvideo" command which, as you can imagine, restarts the video with the current settings :).

Axis events (ie. mice) are also handled in a similar manner, except the mouse deltas are passed to the console command as parameters.

ANYWAY, I chose not to do a pure message system in my new version of the engine because I find they tend to be less deterministic (anyone anywhere can inject a message into the system in any order at any time ^___^) which means they are trickier to debug (potentially :P). They are, however, probably more elegant in the long run (I do think tho that it would be even more elegant in languages other than C++, such as Python or C#).

If you want to go the whole hog, you might want to check out superpig's Enginuity article which covers making a "process" based engine (http://www.gamedev.net/reference/articles/article1947.asp) although you've probably already seen it.




Wow it's almost scary how close his system behaves to the one I have designed for my system. After toying with a bunch of ideas the listener class seems to be the simplest route for me to pursue especially with the entire event system being multithreaded. The event manager now allows a message to be broadcast throughout the system syncronously or asyncronously (and each listener class can decipher this as well since they handle dispatching to each event handler) but I'm still messing with the idea of components being able to broadcast messages internally to themselves without having to go directly through the queue (since some of my events would only be handled by the module generating them).
Quote:Original post by Moe
Quote:Original e-mail from cow_in_the_well
...A win32 style MsgProc type messaging thing is the simplest way to go about things, but I prefer to go about things in more of a C++ manner.


For an example of how to easily turn a MsgProc procedure into a C++ 'pump', you can see http://www.codeproject.com/library/DWinLib.asp#winBaseO. You would probably turn 'WinBaseO' into 'EventObject' or something like that, to modify it for your needs.

Your problem description seems a little lacking, though. For instance, will some keystrokes move the current player and some keystrokes activate 'engine' routines, such as saving the game? Will some mouse events activate the player while others save the game? You probably don't want to broadcast all events to all entities, as that is kinda wasteful. Figure out a way to send the events to where you want them handled, in the simplest manner possible. I suppose you are getting the events from your system (Windows, Unix, ...), so you must build your system with that in mind as well.

It sounds like a fun problem, so enjoy it!

This topic is closed to new replies.

Advertisement