Jump to content

  • Log In with Google      Sign In   
  • Create Account


Member Since 03 Jun 2003
Offline Last Active Yesterday, 04:03 PM

#5159644 MMOs and modern scaling techniques

Posted by hplus0603 on 10 June 2014 - 08:05 PM

modern online games are already using web scaling approaches

There really are two kinds of things going on in most games.
One is the real-time component, where many other players see my real-time movements and my real-time actions.
The other is the persistent game state component, which is not that different from any other web services application.
The trick is that, if you try to apply web services approaches to the real-time stuff, you will fail, and if you try to apply the real-time approach to the web services stuff, you will pay way too much to build whatever you want to build.

Some projects have tried to unify these two worlds in one way or another (for example, Project Darkstar) but it's never really worked out well. In my opinion, doing that is a bit like trying to unify real-time voice communication with one-way streamed HD video delivery -- the requirements are so drastically different, it just doesn't make any sense to do that.
That analogy isn't entirely bad: One-way streamed video is a lot more like "web apps" than real-time voice communication, which is more like interactive games, except without the N-squared interactions and physics rules.

So, one way of approaching the scalability problem is to dumb down the gameplay until it's no longer a real-time, interactive, shared world. That doesn't make for fun games for those people who want those things. The other way is to be careful about only building the bits that you really do need for your desired gameplay, and optimizing those parts to the best of the state of the art.
Getting the game developers, and the web developers, to understand each other, and work under the same roof, AND making sure that the right tool is used for the right problem, while delivering the needed infrastructure for gameplay development, is really hard to pull off; I've seen many projects die because it misses one of those "legs" and doesn't even know what it's missing. (A bit of the Dunning-Kruger effect, coupled with "I've got a great hammer!")

#5159530 game's protocol between client/server

Posted by hplus0603 on 10 June 2014 - 10:14 AM

In general, you want your code to be as table-driven and data-driven as possible. If you find yourself writing the same code multiple times, find a way to compress it into a table lookup or similar.

Thus, the creation of various Actions from string might look like:


class Action { /* abstract base class */ };
class MoveAction : public Action { /* concrete class */ };

class KillAction : public Action { /* concrete class */ };

class DieAction : public Action { /* concrete class */ };

struct NamedAction {
  char const *name;
  Action (*makeAction)(JSON *, Context *);
g_allActions[] = {
  { "move", &MakeMoveAction },
  { "kill", &MakeKillAction },
  { "die", &MakeDieAction },
Action *ActionFromJSON(JSON *json, Context *ctx) {
  char const *type = json->KeyAsString("type");
  if (!type) return nullptr;
  for (size_t i = 0; i != countof(g_allActions); ++i) {
    if (!strcmp(g_allActions[i].name, type)) {
      return (*g_allActions[i].makeAction)(json, ctx);
  return nullptr;


In a larger system, you'll likely use a hash table instead of a linear table scan, and each action kind will register itself in this table on start-up in some way.

#5159528 MMOs and modern scaling techniques

Posted by hplus0603 on 10 June 2014 - 10:08 AM

If I remember correctly, City of Heroes actually would spin up additional instances of "the same area" when player counts got too high. We also did that in There.com, to support large parties. This is good from a player point of view, too; you'd rather see all the players you can interact with, than having 1000 players in the same area and you can only see the nearest three meters...


When it comes to distributed simulation with scalable player density (not just player count,) that's a much tougher nut to crack. Web applications do not have nearly the level of coupling between different business objects that games do. When researching this back in the day, the best option I could come up with would be one where objects are only allowed to affect other objects one tick into the future, and there would be a big cross-bar of messaging of "these are all the interactions I detected this turn" that would be exchanged between the simulation servers between ticks. There would still be a n-squared scaling problem between servers, but with 40 or even 100 Gbps local networking, you can get pretty high in numbers before that becomes a bottleneck. This architecture lets objects be homed on a particular server, and then roam the entire world (assuming the server can keep the static world in RAM) and interact with all other objects with a one-tick delay. This is also very similar to the original DIS distribution model, except DIS typically had a single entitly (plane, tank, etc) per network node, rather than, say, 1000 entities per node.


The other problem is that dense simulation IS an N-squared problem. If everyone piles in a big dogpile, you will potentially have interactions between all players to all other players. There's no way around that. Same thing as collision detection -- in the worst case, every object really DOES interact every other object.

#5159178 Peer to Peer Online Game

Posted by hplus0603 on 08 June 2014 - 09:49 PM

I just don't want to have to send position updates for all monsters to a lot of players from one machine, so I'd have to come up with a deterministic movement simulation.

What problem are you actually trying to solve?

I e, can you provide X here? "Right now, PvE multiplayer games suffer because X."
And can you provide Y here? "My solution solves X by doing Y."

#5158774 How to keep track of sessions in HTTP requests

Posted by hplus0603 on 06 June 2014 - 02:03 PM

Actually, I split this to a separate topic, because it's a different question!


Generally, when dealing with sessions, you use a data store with timeout or time-to-live of some sort. Examples include memcached, Redis, or Cassandra.

For HTTP, when a user logs in, you create a new session and identify it with a session ID. Use a strong random number and verify that it's not already existing. Store information about the session in your data store, and store the session ID in a cookie in the browser.

When you receive HTTP requests with a session ID cookie, look up that ID in your data store; if it's still there, the session is valid.

The session ID needs to be hard to guess, and you need to have billions more session IDs than you have active sessions to defend against guessing attacks, but that's easy with a 128 or even 256 bit strong random number as your session ID, coupled with not allowing more than a dozen bad logins or bad sessions from the same source IP in some amount of time (say, 5 minutes.)


PHP specifically has some session management built in. By default, it just stores the data in a local file, so it only works on a single machine; you can extend it to use memcached or whatever once your service outgrows a single server. But it's also pretty reasonable to build your own as described above.

#5158420 Preventing cheating on game's server side

Posted by hplus0603 on 05 June 2014 - 09:37 AM

If you process the next message in the input queue as soon as it is there, and discard messages that are against the rules, then the "cleaning" will happen by itself, because if someone sends a second message, that will immediately be discarded (because it's against the rules.)

#5157243 multi-player games and physics engines

Posted by hplus0603 on 31 May 2014 - 06:30 PM

Yes, there is extra cost to re-playing physics.


Sometimes, you can store the state of each player at each step in the past for some amount of replay log, and only replay the entities that got invalidated. If entities interact (collide, weapon hit, etc,) then those sets of entities need to be rolled forward from the time when they interacted.


Often, physics simulation isn't particularly expensive, so the cost to doing this is acceptable, especially for fast-paced shooters with less than a hundred players per level.

Once the player count goes up, it starts becoming more important to hide latency in other ways, either by simulating commands when-received, or forcing players to lead shots, or something else.

#5155751 Packet combining

Posted by hplus0603 on 24 May 2014 - 05:35 PM

How about the "some packets to player A, some to player B" issue? How is that dealt with?

1. Set a network tick rate, such as 10 Hz or 20 Hz or 60 Hz
2. Each time a network tick comes up, iterate over all connected players
3. For each connected player, glom together all the updates that that particular player needs to see, and send as one packet to that player

#5155131 Multiplayer game architecture question

Posted by hplus0603 on 21 May 2014 - 04:19 PM

It's impossible to tell based on your description. As long as you solve synchronization such that you never get deadlocks, you're probably OK. Also, you don't want to be spawning a new thread each time you have anything to send to the server, but rather re-use a worker thread that pulls data to send from a queue.

Also, you generally don't want to be notifying more than one piece of the application about incoming data. Instead, it's common to use a model-view-controller system where the network acts as controller, and affects the model; the view then gets notified by the model when data changes. One set of data might be "whose turn is it?" and when that changes, the view changes how it displays itself.

#5154657 Using weak server side hash's.

Posted by hplus0603 on 19 May 2014 - 11:16 AM

The first question is: What kind of attack are you trying to defend against?
The second question is: What methods already exist for defending against that attack?
The third question is: How is this method different, for both good and bad purposes?

For example, if you calculate sha1 on the client, then that means that knowing the sha1 of the password is sufficient to log in -- you don't need to recover the original password, because the only thing the server sees is the sha1. You have effectively just changed the password to be a known 160-bit pattern.

Similarly, if you use hash on the server, then you do that to defend against the risk that the server database leaks. The two kinds of exposure that leads to is making it possible to log in to your service, and using the same password to log in to another service as the same user. Weak hashes allows the attacker to construct a duplicate for logging into your service much easier.

#5154170 Multiplayer game architecture question

Posted by hplus0603 on 16 May 2014 - 07:57 PM

If you are using TCP, then you first create a "listening" socket. This socket is "ready" when some connection is available, and you retrieve a second socket, representing that connection, with accept(). Each time the "listening" socket is ready, you call accept() to generate another socket that represents a new connection. For two players, you need to accept() at least twice.

If you have more than one socket that may become "ready" (have input data or connections available) then you need to use select() to check all the sockets at once. Once select() says that a socket is readable, then one call to recv(), or one call to accept() for listening sockets, is guaranteed not to block.

#5152335 Mob/NPC's AI in a MMO with Unity + Custom Server

Posted by hplus0603 on 08 May 2014 - 10:39 AM

the server doesn't know ANYTHING about the terrain

You likely need to fix this.

There are a few ways to do this. For example:
- You could run the same physics on client and server
- You could pre-process the game level to generate a simplified "walkable mesh" for where the monster can be/see and run a different physics simulation on the server
- You could give each monster a "must not leave area" defined by a rectangle or polygon, and it just stays in that area, and have artists define this area for each monster, and make sure there are no walls in that area

#5151919 Spawn & Control MOBs with a MMO server?

Posted by hplus0603 on 06 May 2014 - 04:17 PM

The very basic "spawner" mechanism looks like this:

There's a difference between "spawners" and "monsters."

A "spawner" object sits in the world, and knows how to create "monsters."

Once a "spawner" has created a "monster," it tracks that monster until it is killed.

When the monster is killed, the spawner sets a timer for some time in the future, where the monster will be re-spawned.

The timeout of that timer is typically determined as some minimum amount of time from time of death, plus some minimum amount of time since previous spawn.

Group spawners can spawn (and track) multiple monsters.

Special spawners only spawn werewolves during full moons and whatever.

So, you create NO monster during server start-up, but instead just create all the spawners, and tell them "world loaded" (or, perhaps simpler, "all your monsters are dead.")

#5151841 Why is it that some say that open source multiplayer games are less secure?

Posted by hplus0603 on 06 May 2014 - 11:02 AM

It is absolutely true that open source lowers the bar for certain kinds of cheats!

Open source also lowers the bar for security minded developers to help prevent cheating. Your job is to get as many secure-minded contributors as you get cheating players!


Separately, even if a game is not open source, a determined cheater will be able to cheat. De-compilation and assembly isn't THAT hard to work with.for a skilled programmer.

The draw-back is that it's much harder for a security-minded developer to help prevent cheating in the closed-source world. The relative incentives are less in favor of the developer.


The choice is not at all cut-and-dried. There's pluses and minuses on both sides. And cheating is only one small part of the giant challenge that is developing an online game -- if open source, or closed source, gives you a significant leg up in some other department (schedule, tools, robustness, cost, etc) then that's quite likely to be the deciding factor, rather than the different profiles of cheating/cheaters/mitigation/help.

#5149990 Preventing code duplication

Posted by hplus0603 on 27 April 2014 - 06:53 PM

Actually, OO in itself is not always the right solution. You could just as well express the same functionality as free functions working on typed data structures, and if you do that, the linker is better at stripping out the parts you don't use.

If you have to use object oriented classes, then you can decompose your objects into multiple smaller classes, and use aggregation to include the "bits and pieces" that you need on server versus client. Thus, you'd have separate "entity factories" on client and server, where the client-side factory created the player object with user-derived input, and created all objects with rendering components, and created all non-player objects with network-derived input. On the server, all objects are created with network-received input, no rendering, and authoritative data output over the network.

In general, if you stick to OO, an object/component structure will let you re-use the parts that make sense between client and server, while attaching other bits and pieces that only make sense on one or the other. As long as you keep a strict interface (pure abstract base class) between the objects/components and their dependencies, this will work out well.

I'd highly recommend the libraries-with-data-structures approach, though. This lets you still re-use code, by calling the appropriate functions on the appropriate data in the appropriate context.