30k concurrent players on a (private) MMO server...is this possible ?

Started by
25 comments, last by Butabee 9 years, 3 months ago

Hi all,

I know that private server is a touchy subject, but please bear with me.

Recently, a work colleague claimed that he has hosted a MMO server which can sustain 30000(!!) players, all are online at the same time.

Furthermore, all the players are in very small number of areas, doing PvP.

I am very perplexed by this claim, and personally feel that it is a very tall claim, but my other work colleagues said it's possible due to the game in question require very small bandwidth.

I would like to hear from you guys on what you think from MMORPG networking technical standpoint. What would be (very rough) estimate of bandwidth and also server hardware required to support such conditions. Is this possible at all ?

Further details :-

- He claimed this is way back in 2005. The server in question is single node powered by Pentium 4

- The MMO in question is Ragnarok Online, and the PvP event is War of Emperium

Advertisement

Hi all,

I know that private server is a touchy subject, but please bear with me.

Recently, a work colleague claimed that he has hosted a MMO server which can sustain 30000(!!) players, all are online at the same time.

Furthermore, all the players are in very small number of areas, doing PvP.

I am very perplexed by this claim, and personally feel that it is a very tall claim, but my other work colleagues said it's possible due to the game in question require very small bandwidth.

I would like to hear from you guys on what you think from MMORPG networking technical standpoint. What would be (very rough) estimate of bandwidth and also server hardware required to support such conditions. Is this possible at all ?

Further details :-

- He claimed this is way back in 2005. The server in question is single node powered by Pentium 4

- The MMO in question is Ragnarok Online, and the PvP event is War of Emperium

If you use dynamic allocation then you can keep adding players until you run out of memory.

The real constraint would be CPU and network of course.

EVE-online has had like 70,000 on a single realm but I do believe they have a cluster of physical machines that behave as one server.

It really depends on how its coded. If the amount of memory used by each player is very small and there isn't alot of CPU overhead then you can have tons of players.

Assuming the server has a 100mbps internet connection, divide that by 30000 and you're left with 416 bytes per second, per player.
It's probably possible to design an MMO architecture where each player uses less than half a KBps of bandwidth... I've never done it biggrin.png but it doesn't sound completely unbelievable.

As far as I know, commonly these games would use 1-3KBps, so 0.5KBps is quite low... but possible. Keep in mind that servers often run at extremely low frame-rates, such as 10fps, or lower!

It's definitely possible to design code to update 100k MMO-type characters on one CPU.

The hosting costs would of course be massive, unless it's hosted somewhere where 100mbps/gigabit access is common...
Also, the server machine would have to have lots of IP addresses assigned to it (or a large IPv6 subnet) else they'd run out of network ports!

Assuming the server has a 100mbps internet connection, divide that by 30000 and you're left with 416 bytes per second, per player.
It's probably possible to design an MMO architecture where each player uses less than half a KBps of bandwidth... I've never done it biggrin.png but it doesn't sound completely unbelievable.

It's definitely possible to design code to update 100k MMO-type characters on one CPU.

The hosting costs would of course be massive, unless it's hosted somewhere where 100mbps/gigabit access is common...
Also, the server machine would have to have lots of IP addresses assigned to it (or a large IPv6 subnet) else they'd run out of network ports!

Yeah, personally the server connection requirement is what caused me to hardly believe this claim. He justified that many players are online and in very small areas because of the weekly PvP siege event, but from my understanding that would even shoot up the bandwidth per player, considering RO PvP has lots of Area of Effects elements.

I read somewhere in this forum that MMORPG that time normally target 1kb/s per player on average, but in this situation it must be way higher than this right ?

I believe even if the server allow and can sustain 3000 (1/10 of his claim) players in one area, I believe the players internet connection might not.

Appreciate more input from you guys, hehe. I really want to believe him, but if this possible at all it must be a huge technical feat and I would be interested to know how ;)

Another thing I forgot to mention - I have a friend making an MMO that can handle over 500000 players off of a single server (Intel i7 CPU, 30mbps connection)... He does it by keeping most of the traffic peer-to-peer. Players discover who is nearby in the world, and then those players communicate directly without need for the server.

I believe even if the server allow and can sustain 3000 (1/10 of his claim) players in one area, I believe the players internet connection might not.

Let's say whenever you click, you send the server a structure describing that command - where in the world you clicked, what you clicked on, and what kind of click command you were giving (move/cast/shoot/etc):
struct PlayerInput
{
  int32 clickPosX, clickPosY;
  uint32 clickedObjectId;
  uint16 commandType;
}; size = 14
A high level player might have something like 300 clicks per minute, which is 5 clicks per second, which makes 70Bytes/s or 0.56kbps player upload.
On the other end, if there's 30000 clients, that adds up to 16.8mbps of download on the server side.

Lets say that whenever the server gets a movement command, it starts moving that player towards the click location. It then sends every player in that local area an update, saying where that character is moving to, and when they will arrive.
This doesn't have to be sent to everyone in the world, just those players who can actually see this character.
struct CharacterMovement
{
  int32 posX, posY;
  uint32 arrivalTime;
  uint32 playerId;
}; size = 16
Lets say that even though there's 30k characters online, you can only see 20 of them at once. If every player is sending 5 movement clicks per second (everyone is spamming), and everyone can see 20 characters, then the server has to send to each client:
20 visible characters * 16 bytes per update * 5 updates per second = 1600Bytes/s = 12.8kbps
However, the server has to send that to 30000 clients, so the server upload = 45.7MB/s = 384mbps

From there you can tweak things -- lets say that the server ignores click-spamming, so only sends at most 1 position update per character per second. In that case, suddenly the client-side download is 2.6kbps and the server-side upload is 76.8mbps.

You still need extra bandwidth for sending player attribute updates (you lost 20HP!), spawning effects/items/projectiles, moving NPC's etc... But this scenario still sounds feasible for clients on 56k modems and a server with a 100mbps backbone!

I don't see why it isn't possible on a correctly developed and tested protocol.

Let's take irc as a example. Freenode server can regularly have upwards of 10k clients on it connected using tcp and back in the day maybe ten years ago irc servers used to hold tens of thousands more (this is already less efficient than the udp protocols used by most games) that use maybe 512bps average. These servers are usually low end machines, bog standard dedicated servers with 100mbps Burstable... So in short... Why not?

Edit: also irc servers have to deal with regular and brutal denial of service attacks, connect floods etc much like game servers so this might be a good analogy... Food for thought...


He does it by keeping most of the traffic peer-to-peer.

Well with that logic the number of players is nearly infinite. You only need a very small amount of traffic to tell players about each other initially, and from that point on the players mainly communicate with each other. And if you aren't too concerned with cheating, you keep all the item/logic client side rather than server side. No item updates, no location updates, no nothing, just a lot of ip addresses and save/retrieve state.

Why do you think PirateBay was one of the world's biggest sites with the least amount of hardware. P2P tech rocks.


Another thing I forgot to mention - I have a friend making an MMO that can handle over 500000 players off of a single server (Intel i7 CPU, 30mbps connection)... He does it by keeping most of the traffic peer-to-peer. Players discover who is nearby in the world, and then those players communicate directly without need for the server.

That is very interesting way to reduce the bandwidth requirement for the server. I think for turn based MMO, or low level players per grid/area it would be great.

In case of real-time combat MMO with high number of players per grid, I think P2P will be way harder especially on maintaining 'chronology authority' ( e.g who attack/kill/loot first ) and also attractive for tampering since most data bypassing the main server.

I think Ragnarok Online client itself use standard client-server model and does not support P2P....thus the 'magic' most likely at server side.

I still lacking details of what type of internet connection used by his server, but he just told me he is using consumer level broadband. In 2005, typical broadband speed is only ~1-2Mbps in my country... sad.png


I don't see why it isn't possible on a correctly developed and tested protocol.

Let's take irc as a example. Freenode server can regularly have upwards of 10k clients on it connected using tcp and back in the day maybe ten years ago irc servers used to hold tens of thousands more (this is already less efficient than the udp protocols used by most games) that use maybe 512bps average. These servers are usually low end machines, bog standard dedicated servers with 100mbps Burstable... So in short... Why not?

Edit: also irc servers have to deal with regular and brutal denial of service attacks, connect floods etc much like game servers so this might be a good analogy... Food for thought...

I think that IRC servers are comparable somewhat ( based on my personal and not professional knowledge :D ). But in a way you making me doubt my friend even more. There is nothing remotely close to 100Mbps in 2005 in my country for non-enterprise user.. maybe around 2-5Mbps only unsure.png

Bandwidth is very unlikely to be the constraining factor, actually.

CPU starvation would be much more probable. A game of any appreciable complexity takes serious computational overhead well beyond the network requirements. RO is a pretty simple game (no 3D physics, straightforward rules, etc.) but I'd be pretty surprised in a 2005-era P4 could simulate 30,000 simultaneous players.

Think of it this way: if it was that easy to host tens of thousands of players, why would MMOs typically sink huge amounts of money into datacenter hardware? It'd be stupid to build all the infrastructure to host your playerbase if you could do it on your average laptop.

Wielder of the Sacred Wands
[Work - ArenaNet] [Epoch Language] [Scribblings]

Have in mind Ragnarok Online is a 2D game, so don't think "30k WoW players doing PvP" level of complexity. Its sprites on a 2.5D background.

You can get some info about the game and the engine on Wikipedia

http://en.wikipedia.org/wiki/Ragnarok_Online

http://en.wikipedia.org/wiki/AEGIS_(Ragnarok_Online)

Apparently, it wasn't very well designed.

And btw, if he is remembering stuff from 10 years back, definitively he is painting it a bit pink at least. Maybe it "worked", but it lagged like fucking hell.

"I AM ZE EMPRAH OPENGL 3.3 THE CORE, I DEMAND FROM THEE ZE SHADERZ AND MATRIXEZ"

My journals: dustArtemis ECS framework and Making a Terrain Generator

This topic is closed to new replies.

Advertisement