Architectural musings

Started by
16 comments, last by ApochPiQ 6 years, 10 months ago
I originally posted this in another game development community and was largely met with confusion.

I will attempt to re-state my case here in hopes that (A) I can get a sanity check on the design principles and (B) I can have a more productive discussion about the design itself.


First, a motivating example.

Suppose we have a game in which there are cameras. A camera consists of a Position and a FocalPoint, both 3-coordinate vectors. For simplicity we'll ignore other attributes.

Now, the game begins creation when there is only need for one camera and one controlling mechanism: player input. But as time goes on, as always happens, the design is iterated upon and more features become desired.

All told, we will end our game's journey with several things wanting to control cameras:
  • Player inputs
  • Networked input (spectator cam)
  • Game replays
  • Pre-scripted cutscenes
  • Dynamic "temporary" cameras that do things like zoom to a target region, show it briefly, then return to player control
In most game engines I've ever seen, the camera object would have some interface like "SetPosition" and "SetFocalPoint" and all of these modules would then take a dependency on the camera.

My proposal is simple: apply some dependency inversion and wrap the notion of a "vector" inside a notion of a "vector source."

Instead of player input, netcode, replays, and cutscenes all pushing state into the camera, they instead hand the camera a "vector source" and say "ask me for your position and focal point whenever you need to."

The bottom line is that suddenly we have no dependencies between modules. Camera controllers can publish their stream of data anywhere - including into a buffer for replays or debugging - and the camera itself only needs to depend on a single interface.


If you're with me thus far, permit me to take this design a step further.

Any time you have a stream of data that changes as the game plays, apply this dependency inversion. Instead of having a push-into mechanism for propagating new state through the game, favor a pull model.

This does not need to imply a performance penalty; for example the camera module could update its position once per frame (by querying its source) and use an internally cached copy of the vector do set up view matrices etc. So really it's about the same as having external code push the value into the camera, just cleaner.


My hypothesis is that uniformly supplying runtime data in this way would make certain things much easier.

Consider the camera above. Once we have a position controller that works on the network, we can apply that vector source to any game object and it suddenly has "free" replication capabilities.

Or think about the way replays work. Any value in the game can be streamed into a replay file with uniform logic; just write an entry to a log every time the value source changes its state. To "replay" the log, just attach a value source to the relevant game object's properties, and the game will suddenly be driven by a replay instead of live data. No extra infrastructure or hard-coded logic necessary.

Now think about debugging. If a stream of values looks wrong, dump it to disk. Or plot it on a graph in UI. Or whatever. But you only have to write the debug monitoring code once. Any other compatible set of data (e.g. another vector-3) can be dropped in without any fuss.



I feel like I've been doing a terrifically poor job explaining this, because so far the biggest reaction I get from people is being offended that I'd suggest departing from ECS or whatever holy architecture is sexy to them.

I personally think it's not much more than a judicious application of age-old design principles (dependency inversion and abstraction) but it feels like an interesting way to make a game.


For the record, I have worked on games that did this (at my urging) and built a few tech demos that were designed entirely this way. I'm pretty happy with it as a strategy but I'm also very motivated to understand the outright hostility the idea gets from others.

Wielder of the Sacred Wands
[Work - ArenaNet] [Epoch Language] [Scribblings]

Advertisement

I am in favor of update-and-push for reasons related to control.

The request-when-needed model at this fine of a level has a few problems.

  • If the source hasn't been updated then you just get the same value back anyway.
  • If the value is dynamic enough that every request to get an update results in a new value being returned, then you have a performance problem and potentially a sync problem.
    • If only a single source is polling the "vector source" then you have exactly the same relationship as if a camera controller were updating a local once and sending that as the reply, and in this case you could just have the camera controller update "its" vector directly on the camera.
    • If many sources are trying to poll the "vector source" then you would want to have a mutable copy to send out for each matching request rather than updating it each time.
      • Now how often the camera controller updates needs to be tightly controlled to ensure consistent results. Each pull should return the same result for the duration of a logical update, or for a single render, etc.
        • By the time you have gotten this far, the relationship between the camera and the camera controller isn't really, "I, the Camera, call upon the vector source to signal to the Camera Controller to update and give me a value." The Camera Controller is updating at its fixed rate, buffering the value, and allowing things to read it when they need, which is how most of everything works. It's exactly the same functionally and performance-wise as the Camera Controller updating its value and writing it to all the Cameras that need it (which again is almost always just 1). Pull requests will keep getting the same value until the Camera Controller updates, so you are back to the first point in this bullet list.

Things have to be tightly controlled for results consistent not just between replays but even inside a frame. You can't have 2 objects poll the time in a frame and get slightly different results or one will end up falling farther than the other.

If the main concern is really about relationships and what knows about what, I implemented a solution for exactly this inside my animations.
A Track knows only what floats or ints or bools are. It doesn't know about vectors or any other types of objects.
You pass it a pointer to the float, bool, or int that it is supposed to update, the Track determines values as it interpolates between key values as you update it each frame, and it updates the value to which the pointer points.

To update a vector, you have 3 tracks pointing to 3 floats. You can have as many tracks as you need to update any property of any object in the entire game, all without creating objects that know too much.

To apply this to your situation, you would attach a Track to each of the floats on your camera and use the Camera Controller or Network or Player Input to update the values the Track writes out. A Track is meant just to play an animation over a series of key frames, so by this point you would want to stop calling it a Track and call it "MyBigBadFloatUpdater" or something.

Your Camera has dirty flags to set when its position etc. changes? No problem. In my Tracks you pass an optional pointer to a flag and a value to OR into it when writing out a value. The Track will write directly to your Camera floats and set appropriate dirty flags as needed.

I prefer this system as everything gets updated when it should in a controlled environment, and nothing knows too much about the outside world. A Camera Controller doesn't need to know what a Camera is (even though it would make sense if it did), and a Camera doesn't need to know about anything else. The Camera Controller just knows about "MyBigBadFloatUpdater" which could be attached to vectors on a camera, or a player, or particle, or even a Hodgman.


L. Spiro

I restore Nintendo 64 video-game OST’s into HD! https://www.youtube.com/channel/UCCtX_wedtZ5BoyQBXEhnVZw/playlists?view=1&sort=lad&flow=grid

If the source hasn't been updated then you just get the same value back anyway.


I legitimately don't know how that is a problem.


If the value is dynamic enough that every request to get an update results in a new value being returned, then you have a performance problem and potentially a sync problem.


That's true regardless of how your plumbing works. Careful order of operations and reliable sequencing are key pillars of game architecture. I don't know why you think this is peculiar to my suggestions.



If only a single source is polling the "vector source" then you have exactly the same relationship as if a camera controller were updating a local once and sending that as the reply, and in this case you could just have the camera controller update "its" vector directly on the camera.


Yes, that is technically true, but you're missing the point of having the directions inverted.



If many sources are trying to poll the "vector source" then you would want to have a mutable copy to send out for each matching request rather than updating it each time.


This doesn't follow. All you do is store the state in a central location (doesn't matter what "object" owns it if you want to be nitpicky) and update it as necessary from the point of authority for that data.

In other words, you publish your data to a storage location, and consumers just query it from there. The frequency of updates is irrelevant and no different a challenge than any other game architecture.


Now how often the camera controller updates needs to be tightly controlled to ensure consistent results. Each pull should return the same result for the duration of a logical update, or for a single render, etc.


Yes. This does not contradict anything I suggested.

By the time you have gotten this far, the relationship between the camera and the camera controller isn't really, "I, the Camera, call upon the vector source to signal to the Camera Controller to update and give me a value." The Camera Controller is updating at its fixed rate, buffering the value, and allowing things to read it when they need, which is how most of everything works. It's exactly the same functionally and performance-wise as the Camera Controller updating its value and writing it to all the Cameras that need it (which again is almost always just 1). Pull requests will keep getting the same value until the Camera Controller updates, so you are back to the first point in this bullet list.


But what you're describing is exactly what I'm advocating for. I'm confused.

The whole idea of this design is that you can decouple things at a code level without compromising control or performance.


Things have to be tightly controlled for results consistent not just between replays but even inside a frame. You can't have 2 objects poll the time in a frame and get slightly different results or one will end up falling farther than the other.


This is starting to feel like a straw-man in all honesty. I didn't say you ask for an update, you ask for a value. Updating the values is something I purposefully left vague because it does require the level of control you're describing.

Apparently in my quest to avoid describing every last detail of a relatively simple architectural decision, I have fallen into the trap of leaving too much open for interpretation.


If the main concern is really about relationships and what knows about what, I implemented a solution for exactly this inside my animations.
A Track knows only what floats or ints or bools are. It doesn't know about vectors or any other types of objects.
You pass it a pointer to the float, bool, or int that it is supposed to update, the Track determines values as it interpolates between key values as you update it each frame, and it updates the value to which the pointer points.
To update a vector, you have 3 tracks pointing to 3 floats. You can have as many tracks as you need to update any property of any object in the entire game, all without creating objects that know too much.


This is shockingly similar to what I'm proposing. The only difference is I would specialize on the case of a 3-vector and you chose not to. Aside from that, we're talking about the exact same concept.


At this point this is echoing the other conversations I've had about the whole thing, which makes me suspect I'm rather shit at explaining myself :-/

Wielder of the Sacred Wands
[Work - ArenaNet] [Epoch Language] [Scribblings]

At this point this is echoing the other conversations I've had about the whole thing, which makes me suspect I'm rather shit at explaining myself :-/

Sorry.
There’s a fine line between requesting something that has been held for you, and holding something to be requested from you.

If you believe that I have described what you wanted to explain with only the modification that you made an object for 3 floats and I suggested handling 3 floats separately, than the we are indeed talking about the same thing.
In my proposal you make a basic object to handle floats and then build on that if you want to handle vectors, but otherwise indifferent.

This is what I have implemented and suggested. If that was not clear, then please explain.


L. Spiro

I restore Nintendo 64 video-game OST’s into HD! https://www.youtube.com/channel/UCCtX_wedtZ5BoyQBXEhnVZw/playlists?view=1&sort=lad&flow=grid

Since this is pretty much impossible to talk about without code...

https://github.com/apoch/scribblings/tree/master/ValueSourceDemo


This is a very skeletal outline of what I'm talking about. It's simple but demonstrates the shift effectively.

You may be inclined to react with "That's it?!" which is precisely what I'm going for. It isn't a huge change at all. It's a reflection of a basic dependency inversion that everyone ought to be familiar with already.

Note how the classic moving object gets an advance pumped into it while the value-source controlled object has a layer of indirected storage for its position.

Hopefully this code will put everyone on the same page as far as what I'm talking about.

Wielder of the Sacred Wands
[Work - ArenaNet] [Epoch Language] [Scribblings]

Maybe I'm misinterpreting, but your explanations and code look an awful lot like functional reactive programming.
Yup, this is basically reactive programming with a uniform abstraction for a data stream so that any stream of the right type can interchange with any other stream.

It's not inherently functional in the sense that you are free to implement a value source in a more traditional imperative advance-loop model.

Wielder of the Sacred Wands
[Work - ArenaNet] [Epoch Language] [Scribblings]

Updated the example source a bit.

Wielder of the Sacred Wands
[Work - ArenaNet] [Epoch Language] [Scribblings]

I believe I get what you're going for, if only because I've had similar musings recently :D

If I may be so bold, perhaps the reason why you're not getting the responses you're hoping for is because you're focused on explaining and demonstrating the mechanics of something, but haven't really described what that "something" actually is or why anyone would want to use it? You've shown what is it to invert the dependencies on objects into dependencies on data, but I think you've stopped short of answering the next logical question, which is "who owns the data source"? And if this example is extended to all game data, then certainly this data has to exist in an organized fashion somewhere. So the next question would be, "what does this system or architecture for storing and manipulating data look like"?

In my mind, what's essentially happening is that you're divorcing the data model from the object model, which allows the two models to exist independently but function cooperatively. Going back to the controller/camera example, conventional wisdom would dictate that each camera object own its own position, and anyone who wants to update or retrieve the position would have to go through the camera. This forces the structure of your data to follow the structure of your objects, which might not always be ideal. But by using a "data source" instead, the camera no longer owns its position and instead just queries some external source, which is also happens to be updated by the camera controller from time to time. In this sense, neither really own the data. You just have objects that manipulate some shared state, to which we've applied the semantic meaning of "camera position".

My own thoughts on the subject lead me to envision some sort of simple hierarchy, where each data "node" can either be an instance of type T, an array of type T, or an association (dictionary) of string to type T. Think JSON, but with the ability for T to also be a pointer to others nodes in the model (for the in-memory representation). The object model can then be created separately, but still reference or "bind" itself to the different parts of the data model. Provide the camera controller with a read/write interface to the position data, and the camera a read-only interface to the position data. If multiple observers want to inspect the position, they too can get read-only interfaces to the position data. If another camera controller comes along and wants a write interface to the position, then the ownership semantics can be handled centrally and uniformly as a feature of the data model. The same goes with push/pull and events/polling. Allow the model to trigger events or callbacks when data changes, and/or for a timestamp or a change flag to be set, and both methods just become opt-in features that allow objects to decide how they want to interact with the model.

Then it becomes clearer how such an architecture could provide you with all the advantages you mentioned in your first post, because everything only has to deal with a single, universal model that's designed specifically around managing data ownership and relationships. This makes replication, serialization, debugging, replays, etc. easier because object relationships and state that's only applicable to the current execution context can be ignored completely.

Hopefully this is somewhat along the lines of what you've been attempting to explain?

This topic is closed to new replies.

Advertisement