Planning Ahead for Multiplayer...
Author
I've come far enough in my longest running game project ever to start worrying about how to design for multiplayer in advance. I know I want to add some "basic" TCP/IP LAN/Internet server/client capabilities at some point, but I imagine I need to care about how I set up my game world in singleplayer first to allow multiplayer extension later. For the record, I'm doing what could be called an RTS, but it's of a less intense kind. If it will be able to handle up to 3-9 clients connected to one host it will be enough.
I'm reading up on TCP/UDP and server/client architecture but I'm nowhere near the end of the beginner phase. Comments would therefore be appreciated regarding my envisioned design:
All data specific to gameplay (that is, excluding graphics and user interface components, there is no sound) should be in the "real" form on the host with clients only holding a not-necessarily perfect copy in their own machines. (Hopefully good enough for client orders sent to host to make sense though.) Host would send clients two types of messages: Critical and non-critical. Critical would be sent through TCP and non-critical on UDP.
Let's take a practical example: Players move around armies on the strategic map. Once they enter battle players can see the progress of those battles, but not actually influence them other than ordering retreat. Else a battle automaticallly progresses until it's decided.
To handle this a TCP message would have to be sent from host to clients that a battle was starting in a specified area, with the specified participants. Once that happens, the battle would progress and information about this progress would be sent through UDP to clients; if a client misses this no big harm done (though he may be annoyed to find his army lost later if lag is bad). The client can on the other hand send a TCP message to the server that he requests to retreat. Once the battle has ended a TCP message with the exact results is sent from server to clients.
Since fog of war is commonplace in this game it's perhaps practical to not update hidden armies all that often to any client. As long as the client doesn't need to perform anything else than prediction it should be alright.
To make for a transparent switch from singleplayer to multiplayer, I would perhaps need separate sets of functions for all game objects. One for client prediction and commands coming from the user interface, and instead a complete set of functions for the server (which is also what should run in singleplayer mode). These functions would obviously also need to construct relevant messages to be sent to clients in event of multiplayer.
Am I too far off or are these decent ideas?
I've never done an RTS, so take my comments with a grain of salt, but I have been doing multiplayer games for a while.
First and foremost, every addressable pawn in the game should have a unique number, to easily identify it across the network. Putting this in your framework early will make things much easier.
For an RTS I would forgo the use of UDP entirely, and stick with TCP. That way you get the reliability of TCP without having to worry about managing and creating a "connected UDP" socket concept. RTS games, are for the most part turn based. Even though unit movement and animations occur every frame, the actual commands being issued can occur in turns.
The server waits for commands from the clients for the current turn. When all commands have been accepted, the server sends out messages to the clients to tell them what to do, and steps to the next turn number.
You probably don't want to worry about doing interpolation and dead reckoning and the like either. With 100s of units in play it would be foolish to send actual positional updates for every unit.
Instead it would be best to incorporate a command queue on the server. If a player selects a group of units, and tells them to move to a certain location, a request is sent to the server with a list of units, and the command (move) as well as the map position.
As long as you keep any random number generators synched up, the behavior should be identical on all machines.
To design singleplayer to easily support this, build the game in 2 sections: The gameserver and the client. Even if these 2 parts run in the same executable, having a distinct API between user input/display, and game logic will be a monumental step towards simple network integration.
i.e.
CClientClass knows when a player has picked units, and issued commands. It queues these commands up for a certain amount of time. When that time expires, it calls functions in CServerClass to process it's commands. CServerClass then Calls back into CClientClass with the results to display (including enemy positions, etc.)
With this model, the only API that needs to change between single player and multiplayer, is the conduit between CClientClass and CServerClass, everything else should pretty much work.
Look at this article also. It describes the network model used for age of empires -
http://www.gamasutra.com/features/20010322/terrano_01.htm
First and foremost, every addressable pawn in the game should have a unique number, to easily identify it across the network. Putting this in your framework early will make things much easier.
For an RTS I would forgo the use of UDP entirely, and stick with TCP. That way you get the reliability of TCP without having to worry about managing and creating a "connected UDP" socket concept. RTS games, are for the most part turn based. Even though unit movement and animations occur every frame, the actual commands being issued can occur in turns.
The server waits for commands from the clients for the current turn. When all commands have been accepted, the server sends out messages to the clients to tell them what to do, and steps to the next turn number.
You probably don't want to worry about doing interpolation and dead reckoning and the like either. With 100s of units in play it would be foolish to send actual positional updates for every unit.
Instead it would be best to incorporate a command queue on the server. If a player selects a group of units, and tells them to move to a certain location, a request is sent to the server with a list of units, and the command (move) as well as the map position.
As long as you keep any random number generators synched up, the behavior should be identical on all machines.
To design singleplayer to easily support this, build the game in 2 sections: The gameserver and the client. Even if these 2 parts run in the same executable, having a distinct API between user input/display, and game logic will be a monumental step towards simple network integration.
i.e.
CClientClass knows when a player has picked units, and issued commands. It queues these commands up for a certain amount of time. When that time expires, it calls functions in CServerClass to process it's commands. CServerClass then Calls back into CClientClass with the results to display (including enemy positions, etc.)
With this model, the only API that needs to change between single player and multiplayer, is the conduit between CClientClass and CServerClass, everything else should pretty much work.
Look at this article also. It describes the network model used for age of empires -
http://www.gamasutra.com/features/20010322/terrano_01.htm
Take the example of Warcraft 3, and what I think it's doing underneath the hood:
I believe Warcraft 3 works by actually always using TCP, and sending the player input to the server, which forwards it to all other players. Everybody then simulate the game in sync (which means that the slowest machine determines the tempo of the game). The draw-back is that everybody will have information about all the enemy units, so there can be "see all" hacks.
In Warcraft 3, when you give an order, there's first a little animation where the units acknowledge the order, and they pack up and get ready to go. Then they actually start moving. This gives the network time to propagate the commands to all machines, so the order takes place at the same time for all machines. If some machine has a large lag spike (bigger than the order latency) then the game will actually be suspended for all other players ("waiting for player X") until the network catches up.
In addition to sending user commands, each client should also send notification to the server when they are done processing step X of the simulation. This lets the server know that everyone can progress to step X+N of their local simulation (where N is the lag compensation factory, thus the unit command delay). If you assume 500 ms lag at worst (client to server to other client), and 100 ms time steps, then every command you give will be delayed by 5 time steps (i e, queued for 5 time steps in the future), and when everybody has reported "completed step 10 input," it's OK for your local simulation to proceed up to step 15 (because you've gotten all the input that pertains to that step).
Note that the lag compensation and time synchronization mechanism is separate from the "everybody only sends inputs" co-simulation mechanism in this case. You could, in fact, send unit states, instead of client commands, and still use the same command queuing strategy to hide network latency.
Last, I would recommend that you pick only one protocol (TCP or UDP) and use that for all the traffic, as using both adds more gnarls in your code, and likely isn't actually necessary.
I believe Warcraft 3 works by actually always using TCP, and sending the player input to the server, which forwards it to all other players. Everybody then simulate the game in sync (which means that the slowest machine determines the tempo of the game). The draw-back is that everybody will have information about all the enemy units, so there can be "see all" hacks.
In Warcraft 3, when you give an order, there's first a little animation where the units acknowledge the order, and they pack up and get ready to go. Then they actually start moving. This gives the network time to propagate the commands to all machines, so the order takes place at the same time for all machines. If some machine has a large lag spike (bigger than the order latency) then the game will actually be suspended for all other players ("waiting for player X") until the network catches up.
In addition to sending user commands, each client should also send notification to the server when they are done processing step X of the simulation. This lets the server know that everyone can progress to step X+N of their local simulation (where N is the lag compensation factory, thus the unit command delay). If you assume 500 ms lag at worst (client to server to other client), and 100 ms time steps, then every command you give will be delayed by 5 time steps (i e, queued for 5 time steps in the future), and when everybody has reported "completed step 10 input," it's OK for your local simulation to proceed up to step 15 (because you've gotten all the input that pertains to that step).
Note that the lag compensation and time synchronization mechanism is separate from the "everybody only sends inputs" co-simulation mechanism in this case. You could, in fact, send unit states, instead of client commands, and still use the same command queuing strategy to hide network latency.
Last, I would recommend that you pick only one protocol (TCP or UDP) and use that for all the traffic, as using both adds more gnarls in your code, and likely isn't actually necessary.
A good way to keep random number generators in sync is just to have the server issue the values. It will take little bandwidth unless you're calling it constantly. Another thing the server can do to stop "see all" hacks is to keep a list of all the units and where they are and only send updates to the clients that can see the units moving.
You should also consider building the single player game as a client/server setup to make your life easier.
You should also consider building the single player game as a client/server setup to make your life easier.
Like anything in life, there is a tradeoff between security and flexibility:
If you have a dedicated server, and that server will never change, then sending only information pertinent to the client is possible. However, if you want to support server migration when the server drops, so that other players can continue with the game, you run into trouble:
Lets take this example:
1. in a 3 player game, player 1 is the server.
2. five minutes into the game player 2 zerg rushes player 1 and wipes out his entire army.
3. Player 1 is dead.
4. Player 1 turns off his computer in a fit of fury.
5. player 3 is determined to be the new server.
6. Player 3 does not know anything about the state of player 2, and cannot accurately simulate the game without knowing the complete and accurate gamestate of player 2. (Or the position of any structures that player 1 may have built, such as fences, etc, damaged terrain, etc..)
Even though distributed gamestate insecure, it's equally insecure to all players. If the server is the only machine that knows the gamestate, the player with the server could still cheat, whereas none of the other players could.
That's more of a philosphical argument, but I think it ressonates well with most gamers.
If you have a dedicated server, and that server will never change, then sending only information pertinent to the client is possible. However, if you want to support server migration when the server drops, so that other players can continue with the game, you run into trouble:
Lets take this example:
1. in a 3 player game, player 1 is the server.
2. five minutes into the game player 2 zerg rushes player 1 and wipes out his entire army.
3. Player 1 is dead.
4. Player 1 turns off his computer in a fit of fury.
5. player 3 is determined to be the new server.
6. Player 3 does not know anything about the state of player 2, and cannot accurately simulate the game without knowing the complete and accurate gamestate of player 2. (Or the position of any structures that player 1 may have built, such as fences, etc, damaged terrain, etc..)
Even though distributed gamestate insecure, it's equally insecure to all players. If the server is the only machine that knows the gamestate, the player with the server could still cheat, whereas none of the other players could.
That's more of a philosphical argument, but I think it ressonates well with most gamers.
Author
Okay, so what I gathered from your posts:
Assign each unit/element/piece an identifier: Actually I was already doing this, but it's good to know I'm doing the right thing. Some identifiers are indices into static areas (of course bounds checked), and some refer to dynamic elements such as the armies I was speaking about.
Use one protocol: I'll go with TCP then because communication shouldn't have to be very intense due to the game's nature.
I won't support server migration: I'll just be convenient and say "set autosaving on the server to the most frequent interval".
I'm a bit confused about the client/server concept though (I know what it's defined to be but still). Who's responsible for holding the game world data? Some other storage class (CWorld is what I had until now) with client/server code only interfacing to this data (both sharing in case of singeplayer, else there is just one), or should the server keep a separate copy of the world separated from the client?
It's not decided also whether I should make synchronization across multiple machines (although it is what you seem to recommend for an RTS). My counter-reasoning is the client in the case of multiplayer should be able to maintain a reasonable state of the world merely by keeping track of object additions and deletions, and frequent updates of those that are visible. (This game only features a low number of heavily dynamic objects. Most objects just sit in place until you do something to them.)
Assign each unit/element/piece an identifier: Actually I was already doing this, but it's good to know I'm doing the right thing. Some identifiers are indices into static areas (of course bounds checked), and some refer to dynamic elements such as the armies I was speaking about.
Use one protocol: I'll go with TCP then because communication shouldn't have to be very intense due to the game's nature.
I won't support server migration: I'll just be convenient and say "set autosaving on the server to the most frequent interval".
I'm a bit confused about the client/server concept though (I know what it's defined to be but still). Who's responsible for holding the game world data? Some other storage class (CWorld is what I had until now) with client/server code only interfacing to this data (both sharing in case of singeplayer, else there is just one), or should the server keep a separate copy of the world separated from the client?
It's not decided also whether I should make synchronization across multiple machines (although it is what you seem to recommend for an RTS). My counter-reasoning is the client in the case of multiplayer should be able to maintain a reasonable state of the world merely by keeping track of object additions and deletions, and frequent updates of those that are visible. (This game only features a low number of heavily dynamic objects. Most objects just sit in place until you do something to them.)
Quote:should the server keep a separate copy of the world separated from the client?
In my opinion, it absolutely should with the model that you suggest. It doesn't have to in the Warcraft-3-like setup I previously described, because all clients (including the client that's serving) will have the exact same game state.
You can decide to send object creates, deletes, and occasional updates if that's what you want to do -- it's certainly simpler than getting co-simulation to work right. The main draw-back is that whomever is hosting the server can cheat in ways the other clients can't, because it can send authoritative updates about the entity states. Plus bandwidth -- entity updates will take bandwidth.
Author
Quote:Original post by hplus0603Quote:should the server keep a separate copy of the world separated from the client?
In my opinion, it absolutely should with the model that you suggest. It doesn't have to in the Warcraft-3-like setup I previously described, because all clients (including the client that's serving) will have the exact same game state.
Okay, then I'll work with some kind of "displayable world" instead that's closer connected to the graphics output (alright since I needed that an inteface to draw world entities in the first place).
Quote:You can decide to send object creates, deletes, and occasional updates if that's what you want to do -- it's certainly simpler than getting co-simulation to work right. The main draw-back is that whomever is hosting the server can cheat in ways the other clients can't, because it can send authoritative updates about the entity states. Plus bandwidth -- entity updates will take bandwidth.
There won't be all that many entities that need frequent updating so I don't think bandwidth is an issue. Cheating is an issue obviously, but my game is not mainstream and I don't expect people to play with people they don't know. (Instead they can exclude cheaters.)
I have similar design 'problem' for my game; let's say it's a 2D Arcade Space Sim [if you know SubSpace, you'll understand the gameplay idea].
I agree that client-server design for both singleplayer and multiplayer mode is a very good idea, and I'm going to do it that way. I thought about a design using server and client abstract interfaces in the way:
Client gameplay code communicates with server by the Server Interface; it doesn't know anything about where and what the server is.
Server gameplay code communicates with connected clients through Client Interface; it also does know nothing about client's location, etc.
Now, in single player mode there is one real client and one real server on the same computer, doing communications without network interface between them. The fun begins with multiplayer mode. On client's computer the client is real, while under Server Interface is hidden a Proxy Server, which redirects all communications to the host computer. On server's computer, server of course is real, and most [or all clients if dedicated] Client Interfaces has a Proxy Client under them, which handles communication with other computers.
So; that's my idea, but I'm not sure about this design of communication between server and client - maybe it's too complicated. I've never designed multiplayer systems before, so I have no experience.
In this idea [and generally abstracting server and client in both modes] I find another interesting usage. We can make the Proxy Server also save input to a file, and thus creating a replay saving system for the game.
TeMPOraL.
I agree that client-server design for both singleplayer and multiplayer mode is a very good idea, and I'm going to do it that way. I thought about a design using server and client abstract interfaces in the way:
Client gameplay code communicates with server by the Server Interface; it doesn't know anything about where and what the server is.
Server gameplay code communicates with connected clients through Client Interface; it also does know nothing about client's location, etc.
Now, in single player mode there is one real client and one real server on the same computer, doing communications without network interface between them. The fun begins with multiplayer mode. On client's computer the client is real, while under Server Interface is hidden a Proxy Server, which redirects all communications to the host computer. On server's computer, server of course is real, and most [or all clients if dedicated] Client Interfaces has a Proxy Client under them, which handles communication with other computers.
So; that's my idea, but I'm not sure about this design of communication between server and client - maybe it's too complicated. I've never designed multiplayer systems before, so I have no experience.
In this idea [and generally abstracting server and client in both modes] I find another interesting usage. We can make the Proxy Server also save input to a file, and thus creating a replay saving system for the game.
TeMPOraL.
Don't forget that UDP has another huge advantage which hasn't been mentioned yet. NAT punch-through is nearly impossible with TCP.
It may or may not be an issue in this case, but it's definitely worth considering early on. Of course if you're willing to use an external networking library it should be fairly easy to switch later on..
It may or may not be an issue in this case, but it's definitely worth considering early on. Of course if you're willing to use an external networking library it should be fairly easy to switch later on..
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
