Jump to content

  • Log In with Google      Sign In   
  • Create Account

Gathering updates before transmitting across network


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
5 replies to this topic

#1 Kylotan   Moderators   -  Reputation: 3338

Like
0Likes
Like

Posted 28 May 2012 - 11:26 AM

Say you have some entities that you want to keep synchronised across the network - they might be players, NPCs, etc. On a regular basis some of their properties will change and you need to send out updates to all interested clients. I'm interested in which different methods people use to facilitate this.

So far I can think of three approaches, which are certainly not 100% distinct but which represent different ways of looking at the issue.
  • Manual, by timer or event - whenever it needs to, the server can form a network message containing the entity ID and a list of all the relevant properties, or specific messages for different kinds of changes (eg. a PlayerMoved message with a new position). This can be error-prone if the coder misses an important property change, and it's not trivial to revert changes half-way through a failed operation (which sometimes is useful, especially when doing multi-stage updates based on remote server or database values). However it does allow for very efficient messages (as specific messages for different changes don't need to contain property identifiers or the like), and probably works well for shooters and other games where entities have little mutable state but need to send it often.
  • Dirty flags - the server-side routine alters the entity as necessary, with changed properties having a 'dirty flag' set. Then at the end of the routine, or at regular intervals, the server inspects the entity to see which properties are dirty, gathers them into a property change message, and resets the dirty flags. One problem with this approach is that although you usually need to wrap your objects with whatever facilitates the flag, and it's usually hard to catch every single modification unless you have a very comprehensive wrapper (eg. in C++ changing a value in an std::map won't be picked up unless you have wrapped every possible way of accessing it by key or by iterator)
  • Transactions - the server-side routine makes changes to a logical copy of the entity, and when the routine successfully completes, the changes can be committed to the local entity and also put into a property change message. This makes it easy to roll-back routines that don't complete properly, and makes it theoretically easy to do some distributed or concurrent updates with optimistic locking (although that's not always worked well in practice, apparently). It's apparently not well-suited to games that broadcast data at different rates for different entities (eg. you might send fewer movement updates for distant entities - but their changes are committed as soon as they happen, so when does the message get sent?). It is also awkward to code, especially if you want to allow a non-transactional interface alongside the transactional one, and you have a similar issue to dirty flags in that it's hard to ensure that all changes in a nested structure are observed correctly.
So, I don't have a question as such, but basically I'd just like to hear some discussion about which methods people use (especially for more complex games such as MMOs and RPGs), whether I've missed some useful approaches above, whether there are maybe software patterns that make the transactional or dirty flag approaches easier and less error-prone, etc.

Sponsor:

#2 clb   Members   -  Reputation: 1789

Like
2Likes
Like

Posted 28 May 2012 - 01:28 PM

We use the method you describe as 'dirty flags'. For us, the drawback you describe does not apply. The synchronizable data is structured into attributes, which are changed through a single well-defined interface (IAttribute::Set<T>) and it's not possible to miss setting a dirty flag. In our case, it was not encountered as a burden or being particulary difficult. The server has a constant network rate (typically 20Hz, but configurable in the engine), in which it reads all changed variables and sends them to network in well-coalesced datagrams. This is nice since it disallows multiple individual small datagrams from going out, and keeps the datagrams/second rate well capped (most often to 20/second, if everything fits in one datagram). We haven't encountered much issues with this design, and I don't currently think I'd like to do it any other way. To augment these state sets, we use RPC-like commands what we call 'actions', to provide things like 'player fired', or 'car exploded'.
Me+PC=clb.demon.fi | C++ Math and Geometry library: MathGeoLib, test it live! | C++ Game Networking: kNet | 2D Bin Packing: RectangleBinPack | Use gcc/clang/emcc from VS: vs-tool | Resume+Portfolio | gfxapi, test it live!

#3 ChristerSwahn   Members   -  Reputation: 198

Like
1Likes
Like

Posted 28 May 2012 - 02:36 PM

In the MMOG I'm working on there is a distinct update data feed for each client, since the server is authoritative on all simulation and on what the client/player is able to see. This is done in a two-step process:
1) The simulation engine feeds the client session host with updated game world state.
2) The session host then feeds each client with the player's filtered view of the game world.

This architecture also decouples the simulation engine from the client IO component, and it is fairly straight-forward for the latter to both support global data throttling in case of high server load and individual client data throttling depending on connection quality and data congestion.

The attibute updates accumulate at the session host, so only the most-recent attribute updates are sent to each client.

#4 Kylotan   Moderators   -  Reputation: 3338

Like
0Likes
Like

Posted 29 May 2012 - 09:25 PM

Thanks for the replies!

clb - Could you describe your IAttribute interface a little? Are you able to handle variable length data in standard form, such as std::vector? In a past project we had to create our own sequence and mapping types so that we could spot access to the internal values and set the flag accordingly

ChristerSwan - I like the idea of a discrete client session host. I have usually implemented this sort of thing more like a filter on outgoing messages, but this requires the simulation to create the messages first, which complicates ideas such as accumulating the data before sending. However it does mean I don't need to store separate copies of simulation data for each client's view, which sounds like it could be tricky - are you pretty much storing a copy of a subset of the universe for each player?

#5 ChristerSwahn   Members   -  Reputation: 198

Like
1Likes
Like

Posted 30 May 2012 - 01:06 AM

The main rationale for a separate client session host component is actually scaling. In this MMO the client IO is by far the most resource intensive process, and with this architecture the session host machines can be added according to demand (and they don't have any persistent state so they're more-or-less hot-swappable, the affected players will just need to login again).

Each session host does cache the up-to-date physical universe state with which it can serve all its players (so clients' ad-hoc data requests don't affect the core server). It doesn't "require" per-client cache - but there actually is a small one to facilitate the reducing of the data transmitted to the client (enables keeping track of which data (that is visible to the client) that has been modified).

#6 clb   Members   -  Reputation: 1789

Like
1Likes
Like

Posted 30 May 2012 - 02:26 AM

Our scene layer is three-leveled: scene consists of Entities, which comprise of Components, which have instances of IAttributes. Most components are statically structured, which means that the set of IAttributes that they contain is specified at compile-time. There is special support for DynamicComponent, a component type which allows runtime addition and deletion of IAttributes contained within.

IAttribute itself cannot not contain sequences of elements. If we need vector-like structures, we typically create a new DynamicComponent into the entity in question, and add the sequence/collection of IAttributes into the single DynamicComponent. Scripts can access the Entity-Component-Attribute hierarchy to set up and operate on the data structure.

I could find two screenshots of the editor (don't mind the content displayed, the screenshots were made to show bug reports): http://dl.dropbox.com/u/40949268/AvatarOrientationFixUp.png and http://dl.dropbox.com/u/40949268/Tundra/Tree.png

The technology is called 'realXtend Tundra', and it is open source, if you want to dig into the codebase. Downloadable binaries at http://code.google.com/p/realxtend-naali/downloads/list , and source code at https://github.com/realXtend/naali .
Me+PC=clb.demon.fi | C++ Math and Geometry library: MathGeoLib, test it live! | C++ Game Networking: kNet | 2D Bin Packing: RectangleBinPack | Use gcc/clang/emcc from VS: vs-tool | Resume+Portfolio | gfxapi, test it live!




Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS