Hello everyone!

First I'd like to be transparent and say as this is my first time posting on these forums (although I have been an avid reader!), so please let me know if I did not follow any posting guidelines that I might have missed.

I'm a software engineer who has been pursuing game development as a hobby for the past 3 to 4 years. I'm currently exploring game networking for a fast-paced 2D shooter game. While I've delved into various aspects of game dev, networking is a new challenge for me. I'm using an ECS architecture with Typescript/HTML canvas for the client and Typescript/node.js for the server, employing WebSockets for communication.

I'm facing an issue integrating WebSocket initialization and events management with ECS. My server initiates the WebSocket server and, on a new connection, a function handles initializing the network event callbacks. A significant amount of functionality is within this single function (e.g. calls to world.createEntity and world.destroyEntity), which seems inefficient. My challenge is determining where to process WebSocket messages within the ECS structure.

As an example, let's assume that the following JSON message is sent by a game client to the server:

{
	"type": "command",
	"frame": 54,
	"payload": {
		"moveRight": true, 
		"fire": true
	}
}

Each of my player entities wear a CommandStreamComponent that contains the past X frames of input that the server can use as it progresses through its simulation. This component looks a little like this:

class CommandStreamComponent extends Component {
	commands: CircularBuffer<CommandMessage>
}

class CommandMessage{
	frameNumber: number
	command: Command
}

class Command{
	moveRight: boolean = false
	moveLeft: boolean = false
	fire: boolean = false
}

What I am missing here, is the connection between the WebSocket events callbacks (e.g. on('message', …)) and the ECS components/Systems. In other words, I aim for seamless integration where network event messages are smoothly channeled into the ECS workflow for various types of messages.

How can this be accomplished without compromising clarity? How should WebSockets (or any other full duplex communication solution) be integrated into an ECS architecture? I'm not religiously complying with pure ECS principles but aim for a clean networking approach within it. Insights from experienced developers in ECS and/or netcode would be invaluable.

TLDR: Need a clean method for integrating WebSockets and passing network messages within an ECS flow. Some resources I've used while studying games networking include :

• https://www.gabrielgambetta.com/client-server-game-architecture.html​
• https://fabiensanglard.net/quake3/network.php​
• https://snapnet.dev/blog/netcode-architectures-part-3-snapshot-interpolation​
• https://developer.valvesoftware.com/wiki/Source_Multiplayer_Networking​
• https://docs.unity3d.com/Packages/com.unity.netcode@1.0/manual/index.html​
• Overwatch Gameplay Architecture and Netcode: https://www.youtube.com/watch?v=zrIY0eIyqmI​

I see!

Fore sure there's an argument to be made about keeping the game logic in the ECS and the rest outside. This is actually what I started doing originally for networking (although it's more of a collection of functions at the moment than an actual network manager), and also what I am doing with UI (systems publish events, UI elements subscribe to them). It works relatively well.

However, my concerns about a centralized NetworkManager are the following:

• A centralized NetworkManager which would handle all network related operations would introduce tight coupling between it and several Systems/Components, which is what I am attempting to avoid with ECS, by having small components and small Systems that mostly do one thing. It makes testing a bit more cumbersome as well, as I'd have to mock it for various systems,
• I am also concerned about an event-based manager which would perform updates to the components in the game, as I would not have much control over when these updates happen in the server tick lifecycle. This seem even worse for the creation or deletion of entities.

I actually thought today of simply having a lightweight NetworkEventListener which would push the network messages in a message queue, which would itself be consumed by a NetworkMessageSystem in ECS, exactly when I need them to be dequeued. I would still need to mock the Network messages queue to unit test the NetworkMessageSystem, but that would not really be an issue as it'd be done in one place only.

Thanks for the insight! ?

frob said:
While you linked to a bunch of articles, I'm not sure you've deeply understood what they say.

Indeed. I do understand a fair bit of these articles (I believe so, at least), however this is still mostly theoretical for me as I am implementing these concepts for the first time. It is likely that I misunderstood various aspects, but I'm a point where iterating on an implementation is necessary to grasp what I may have missed.

Before I continue answering, I would like to add a precision: my issue here is more related to binding netcode to an ECS architecture than client-server communication. This is related, obviously, but in other words my issue is about integrating ECS with client-server communication, rather than client-server communication itself.

frob said:
Can you explain more what you mean here?

For sure! Let's simplify a bit and assume a PlayerMovementSystem in the server. This PlayerMovementSystem would query all entities with various components such as colliders, transform, etc. but also a PlayerCommandsComponent, which contains a buffer of player commands for the past x client ticks (future simulation ticks on the server). On the other hand (still on the server), I start the WebSocket server more or less as follows (note that I removed several things including error handling to avoid cluttering the page, and using JSON for prototyping):

const wss = new WebSocketServer({ port: 8080 })

wss.on('connection', (ws) => {
    handleNewConnection(ws)
})

function handleNewConnection(connection: WebSocket) {
	// react to the new connection
	
    connection.on('message', (message) => {
    	const data = JSON.parse(message.toString())
    	// do something with the message
    })
    
    connection.on('close', () => {
    	// react to closed connection
    })
}

Put separately, these parts are relatively straightforward. What I am struggling with, is deciding how and where to channel these network messages to populate the PlayerCommandsComponent buffer. Commands would not be the only type of message I'm interested in, obviously, but this is a starting point.

I hope this clarify my question! ? I will answer your other points bellow.

frob said:
Most (but not all) network systems work best with state-based events. It is far less data to send a message “player 1 is now walking” than it is to send a message every simulation tick that position has been updated. The first done well generates a single message that can be a less than a byte, the second done badly can generate a kilobyte of updates every second.

I definitely agree with that, although it seems more related to optimization than architecture, as the ECS Systems use the same data (it's just the way client and server communicate that changes). I might be wrong here, so feel free to correct me.

frob said:
Creation and deletion of entities should happen relatively infrequently, but there are events for those as well. In many designs there are two ID's, one is the network ID and the other is the local ID, which can be different. The authority indicates that an object needs to be created and has a network ID, along with its serialized contents. When the authority destroys an object, it can just pass along the ID of the object to be destroyed. All the manipulation and updates to the entity would use the network ID, the game client can keep a mapping such as ID's to local object pointers.

This is clear, and I agree of course. This is what I am already doing. Each entity on the server has an entity id that is local to the server (same thing on the client), but each network entity gets assigned a network id (connectionId, although the name is subject to change) by the server. However, unless I am misunderstanding, this is more related to client-server communication than internal server code architecture.

frob said:
Treat network communications as an asynchronous serialized event bus. Create as many channels within that bus as you want by giving a channel ID. Generally there are three you need: Unreliable unordered are continuous status updates you don't care if they're lost, they'll almost always get through eventually. Reliable unordered your network system will keep retransmitting until acknowledged, they'll get there eventually but you're not picky about order. Reliable ordered is a data stream, any communications issues will block it until gaps are retransmitted but you know it will arrive in order.

Interesting! This is an area I need to dig into as I am quite inexperienced with UDP implementations.

Thank you for your precious feedback! ?

Regarding binding network subsystem to entities:

First, there is only one network, in the same way that there is only one “fetch” implementation and only one DOM – it's a physically limited, singleton concept.

I assume that you also have only one scene graph, and only one simulation world (for collision and such.) The network is a singleton just like those.

Second, the way to route messages, is typically to use some kind of addressing. If every entity has a unique ID, and every component within the entity has a unique-to-the-entity ID, then a message can be routed to the right entity and component simply with the path “entity-id, component-id.” Thus, the framing of each message should contain that tuple, and then the innards of each message only needs to be known by the component receiving it.

Third, the connection between an entity and the network, could be an entity responsibility, or could be a special component you attach to an entity, that knows how to find the entity and its components for purposes of networking. However, given that networking affects pretty much everything in a game, and in gameplay, every component and entity type that wants to have good behavior on a network, needs to “know" about networking in the abstract (do I interpolate? do I correct? Which properties do I have which replicate? Do I need serial consistency, or is eventual consistency sufficient? and so on.) Thus, it might be reasonable to say that every component will just talk to networking directly, although they still have the address “entity-id, component-id.”

Finally, for efficiency, you'll typically sort updates by entity first, and by component ID second, so if the outgoing queue has messages for:

• Entity 3, component A
• Entity 2, component A
• Entity 2, component C
• Entity 3, component A
• Entity 3, component B
• Entity 2, component C

The actual queue in the packet / on the wire will likely look something like:

• 2 entities
  • 2 components for entity 2
    • 1 message for component A
      • the message
    • 2 messages for component C
      • the message
      • the message
  • 2 components for entity 3
    • 2 messages for component A
      • the message
      • the message
    • 1 message for component C
      • the message

I agree with the advice that sending input events and detected mis-matches as seldom-sent events, generally performs better than sending snapshots every frame. You should, however, try to send snapshots in the background on some schedule, as a fall-back to fix up whatever state may end up becoming de-synced. Unless you go all the way and implement fully deterministic replay with command queueing (like RTS games) – but that's very hard on top of JavaScript, because math is way under-specified, and not consistent across architectures, or even across browsers.

@hplus0603 @frob Hey there!

I just went through both of your messages. There's obviously precious insight in both, but I believe I will have to iterate on the implementation to consolidate my understanding. I imagine I could question or challenge some aspects, but that'd require a hindsight I don't have yet. In any case, that exchange was definitely helpful to unblock me. ?

Thanks a lot for taking the time to answer, I really appreciate it! It's gonna take a while and several projects, but hopefully some day I can give back to the community with my own experience!

Kind regards,
marem