# MMORPG Servers question

This topic is 4775 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

Hi, We are exploring the possibility to start the development of MMORPG later this year. Our plans are to serve max. 2.000 people ($5 per month). Maybe that doesnt count much like MMORPG, but it surely isnt _just_ common multiplayer. Besides I dont think we could compete feature-wise with HUGE projects where 150 people work on this. Still, somehow I think that 2.000 is reasonable number that is possible to reach, if at least our graphics shall be great and common gameplay features are implemented. It would be our fourth or fifth game project already so wed have enough experience for basic game development. Besides we should have our graphics renderer quite advanced by that time (if everything goes well). The questions: 1. How many players can usually safely connect to 1 server ? I.e. How many servers/PCs would we need to serve those ~2.000 people ? 2. Are there any commercial libraries (with support) for Multiplayer component of MMORPG (codebase) ? 3. How many people would we need to make this all run 24/7 (i.e. support,shards,network specialists) ? #### Share this post ##### Link to post ##### Share on other sites Advertisement Quote:  Original post by VladRIt would be our fourth or fifth game project already so wed have enough experience for basic game development. Besides we should have our graphics renderer quite advanced by that time (if everything goes well). How many units did your previous games sell? IF they were merely hobby projects that didn't sell then you still have too little experience to write, maintain, and run an MMOG. Quote:  1. How many players can usually safely connect to 1 server ? Thousands. Any less and you've got a crap networking layuer / arch. Quote:  I.e. How many servers/PCs would we need to serve those ~2.000 people ? Depends entirely on how complex your game is; if it's a real-time MMOFPS with lots of time-critical gameplay and large amounts of data pinging around then you'll might need as many as a server per couple of hundred players. However, IMHO that only happens if your architecture is crap, your network programmers are crap, and your game-design is crap. It takes considerable incompetence to be unable to manage 500 players per server, and in most games you should be able to manage 1k. If it's a turn-based combat game with multiple seconds per turn and turn-locked actions then you should be able to manage many thousands per server. To put this into perspective, one of the biggest (by player numbers) online MMORPG's with 3D graphical world, semi-realtime combat (the norm, but NOT quake-style) handles 3,000 players per physical server, and their logic-execution + network-pipeline are both less efficient than they could be (although not by much, yet I think I could squeeze quite a few more players on there) Some here will probably tell you you can't get anywhere near that. Quote:  2. Are there any commercial libraries (with support) for Multiplayer component of MMORPG (codebase) ? Plenty. There's even a website devoted to listing them (you'll have to google for it, something like "MMOG technology List" or something like that), although its' very basic info only. Shameless plug: http://grexengine.com (although the site is extremely plain. Just email sales @ grexengine.com and you can get the marketing info etc emailed to you) Quote:  3. How many people would we need to make this all run 24/7 (i.e. support,shards,network specialists) ? For 2k players teh rule of thumb is you need approx 3-7 full-time staff (developers + support + technicians). I've seen 15k players supported, commercially, by a total staff fewer than 5, but that's exceptional (if you've got an excellent project manager on-board and a cunning game-desing, you could manage this too, but I wouldn't recommend it). i.e. you need a minimum (always) of one CSR, one dev (adding new features), and one maintenance (bug-fixer). For 10k players, you're aiming for a total staff of approx 5-15, for 50k players you want 8-20, all depending upon your game etc. #### Share this post ##### Link to post ##### Share on other sites Thanks for feedback, here are my answers: Quote:  How many units did your previous games sell? IF they were merely hobby projects that didn't sell then you still have too little experience to write, maintain, and run an MMOG. Id definitely agree with Maintaining and running MMOG, though I disagree with Writing MMOG. As I said, its gonna be less feature-rich than e.g. Everquest2 (at least 10 times less, that is). This is accomplishable in 2 years with small team with graphics renderer (terrain/characters/animation/special effects/Collision Detection) already completed. Quote:  Thousands. Any less and you've got a crap networking layuer / arch. Sounds good considering wed be happy with 2.000 subscribers. Quote: Quote:  I.e. How many servers/PCs would we need to serve those ~2.000 people ? Depends entirely on how complex your game is; if it's a real-time MMOFPS with lots of time-critical gameplay and large amounts of data pinging around then you'll might need as many as a server per couple of hundred players. However, IMHO that only happens if your architecture is crap, your network programmers are crap, and your game-design is crap. It takes considerable incompetence to be unable to manage 500 players per server, and in most games you should be able to manage 1k. Well, its gonna be RPG, so movement isnt going to be that fast as in FPS, still I can imagine that submitting information about 2.000 players each second (or so) is going to be hard to achieve, but compression is my friend (or so I guess). And 1000 people sounds good. That would mean Id need just 2 servers. Thats accomplishable. Quote: Quote:  2. Are there any commercial libraries (with support) for Multiplayer component of MMORPG (codebase) ? Plenty. There's even a website devoted to listing them (you'll have to google for it, something like "MMOG technology List" or something like that), although its' very basic info only. Shameless plug: http://grexengine.com (although the site is extremely plain. Just email sales @ grexengine.com and you can get the marketing info etc emailed to you) Thanks for links, Ill check those out (some of them I already checked). Naturally I googled already, I just hoped someone will provide a real-world feedback for the library they work with (e.g. like you). Quote: Quote:  3. How many people would we need to make this all run 24/7 (i.e. support,shards,network specialists) ? For 2k players teh rule of thumb is you need approx 3-7 full-time staff (developers + support + technicians). I've seen 15k players supported, commercially, by a total staff fewer than 5, but that's exceptional (if you've got an excellent project manager on-board and a cunning game-desing, you could manage this too, but I wouldn't recommend it). Sounds reasonable, actually thats what I found out elsewhere so far too. So it would be doable on$10k monthly revenue.

Questions :
1. Do you think that its impossible for one programmer to be adding new content and fixing bugs ?
2. How many PCs are required for one server ? I heard stories of 50 PCs per server. Whats that mean ?

##### Share on other sites
you might be able to save on bandwith my doing something like what Shadowbane did, by using the mouse for movement. i thought it was a pretty good idea. Shadowbane required to use as little bandwith as possible, since it was sort of action based, and encouraged huge battles, Player vs Player battles where one guild would storm anothers castle.

a little OT but, the Shadowbane game design was amazing, but in reality it just didnt work. whenever a huge battle would take place, you and 50 people go to storm a castle, all hell (lag) broke lose. the whole castle storming thing was going on in _VERY_ slow motion.

however, that was years ago when the game first came out. id be very interested in knowing if they ever solved this problem.

##### Share on other sites
I don't know what country and state you're in. In California, junior programmers cost at least $5k/month, and people who are experienced go for much more than that. Customer service might be cheaper, but there's still rent, and power, and cooling, and bandwidth, and hardware leasing...$10,000/month wouldn't work at all around here for a company size 3-7.

Regarding players per server, that depends on your degree of security, how much simulation you do, how interactive it is, and all that. If all you're doing is bouncing a few packets a second between players that are close to each other, you can easily go to 2,000 on a single piece of modern hardware. After all, most routers are just regular computers, and do many more packets than that. If you do secure server-authenticated simulation at high simulation rates where each tool, attachment, vehicle, weapon, etc is a fully simulated object, and fully replicated to neighboring servers and clients, and have expensive rules for who can see what, when, and where, then you can cut that in ten. If you're a typical RPG, you can design towards the high end of that. If you're a military simulation, you find yourself at the lower end (but then, your customer might have money to spend on hardware :-)

Quote:
 For 10k players, you're aiming for a total staff of approx 5-15, for 50k players you want 8-20, all depending upon your game etc.

I can second these numbers.

There's also a question of man-hours versus hardware cost. If your people cost you $10k/month, and you can buy a$1k server to solve a problem that would take a person one week to solve, then buying the server makes more sense. I imagine that if your team is based in some country with cheap labor, the equation would look very different -- although what the size of the market would be is then less clear (trying to support the US market from abroad is a tough nut to crack).

##### Share on other sites
Quote:
 Original post by hplus0603I don't know what country and state you're in. In California, junior programmers cost at least $5k/month, and people who are experienced go for much more than that. Customer service might be cheaper, but there's still rent, and power, and cooling, and bandwidth, and hardware leasing...$10,000/month wouldn't work at all around here for a company size 3-7.
Im from Slovakia as can be seen from the header of the thread. This means I can get junior programmers in my area for about $300 per month. Still, if I offer$500 per month, theyll be lining up in front of my door (Im in region with 23% unemployment rate (highest in whole country) where average salary is about $300). True, add 37% to that because of taxes&social insurance, but still its far from$5k as in California. Office holding 10 people costs around $300 per month. Although, Id have to add about 20% to that because of inflation and the fact that it wouldnt start sooner that 2006/2007 IF we shall start the development later this year. Quote:  Original post by hplus0603Regarding players per server, that depends on your degree of security, how much simulation you do, how interactive it is, and all that. If all you're doing is bouncing a few packets a second between players that are close to each other, you can easily go to 2,000 on a single piece of modern hardware. Well, and what about if all 2000 players are close to each other (not necessarily on same screen), but close enough to send packets about all 2.000 players ? How do I scale this into more servers ? Quote:  Original post by hplus0603After all, most routers are just regular computers, and do many more packets than that. If you do secure server-authenticated simulation at high simulation rates where each tool, attachment, vehicle, weapon, etc is a fully simulated object, and fully replicated to neighboring servers and clients, and have expensive rules for who can see what, when, and where, then you can cut that in ten. If you're a typical RPG, you can design towards the high end of that. If you're a military simulation, you find yourself at the lower end (but then, your customer might have money to spend on hardware :-) I guess only time will tell. But I needed to know the worst scenario as to how many severs do I need to buy and setup and maintain. Quote:  Original post by hplus0603There's also a question of man-hours versus hardware cost. If your people cost you$10k/month, and you can buy a $1k server to solve a problem that would take a person one week to solve, then buying the server makes more sense. I imagine that if your team is based in some country with cheap labor, the equation would look very different Could you elaborate more about what do you mean by the example of$1k server ? Like it has software/configuration that prevents some type of crashes/bugs/issues that are otherwise handled only manually ?

Quote:
 Original post by hplus0603although what the size of the market would be is then less clear
Did you mean to say that were just trying to adress home-country market ? LOL, I doubt Id find 5 people here, this is Slovakia with most of people having dial-up being paid by time (i.e. the more youre online the more you pay - pricing per minute). Those $5 theyd pay for 3 hrs of connection during day-time. Although there is pretty strong base of people having broadband (like me). Naturally, since this is MMOG, well try to adress the world market where I think it is possible to find 2.000 subscribers paying$5 each month. I could be wrong though of course. And of course it would have to be new people, since our game would hardly have features that would cause EQ2 players to quit and start here.

Quote:
 Original post by hplus0603(trying to support the US market from abroad is a tough nut to crack).
Well, I think that we could easily cover 18 hours per day of customer service, though 24/7 would be harder to achieve although most probably necessary. What do you mean by supporting US market from abroad ? Answering phone calls ? No problem with language here (except for salary these people require).

I still would like to know what do they usually mean when they say that 1 server is run by 50 PCs. Does it mean Ill have to buy 100 PCs for two servers if I want each server to be able to handle 1.000 players ?

##### Share on other sites
Quote:
 Original post by VladRWell, and what about if all 2000 players are close to each other (not necessarily on same screen), but close enough to send packets about all 2.000 players ? How do I scale this into more servers ?

my guess is simply the game will lag horribly. i have never even heard of (and definetly never played) a MMORPG which could support 2000 players in the same area. Shadowbane, a commercial MMORPG which was designed from the start to support many players in a small area (fighting each other no less) could not handle this. in fact, when i was playing Shadowbane, it could not even handle 1/10th of that many players. even with 200 players the game would lag horribly and be in slow motion. theres really not much you can do about something like this.

##### Share on other sites
Quote:
Thanks for feedback, here are my answers:

Quote:
 How many units did your previous games sell?

...

So. You mean "none", yes?

Sure, it's doable. But, in reality, not unless you have one or more MMOG experts on the team or you have some top 0.001% people in your team (hey, it's possible; just very unlikely).

I don't know your situation, I don't know what mega-genius you have, I'm just pointing out my experiences based on having worked with a lot of people in your position and they all went bankrupt, wound up the company, or simply gave up. And these were the ones with probably the best chances of success, carefully selected from hundreds.

Quote:
 Naturally I googled already, I just hoped someone will provide a real-world feedback for the library they work with (e.g. like you).

Well, obviously I'd say the one I know and use and helped write is "the best", although I'd also say that BigWorld's one is the only other good one (all the rest are moderate to mediocre (butterfly, zona) or just crap (being kind, I'll simply mention no names - it's pretty obvious who the useless ones are when you look at their tech docs).

The other question is: how many of them can you afford? IIRC all the decent ones cost mahola...

Quote:
 Questions : 1. Do you think that its impossible for one programmer to be adding new content and fixing bugs ?

Possible, yes. I've seen it done. However, before you get to 2k people you absolutely MUST have dedicated "content" programmers and "bug-fixing" programmers. Otherwise what happens is one of those two jobs doesn't get done properly...it's jsut too hard, mentally, for one person to handle the workload without getting so depressed they get no work done at all, or simply ignore one aspect.

Quote:
 2. How many PCs are required for one server ? I heard stories of 50 PCs per server. Whats that mean ?

We've been through this: in the above conversation, PC == server.

What you're thikning about are clusters, and I wish everyone would call them that, because that's what they are, they are NOT servers. A cluster is a group of servers joined together to *simulate* a single server, but it is not (should not be) called "a server". Unfortuantely a significant nubmer of people insisit on muddying the waters :(.

By the way - does anyone think it's worth me writing an article on this stuff? I could do a followup to "How to build an MMOG" (I think it's in the FAQ, or else google for it as a Stratics article) which actually contains practical numbers (like the staffing estimates), actual links, resources, etc. A couple of people (including two publishers) asked me to write a *book* on this topic, but I simply don't have *that* much time at the moment...

redmilamber

##### Share on other sites
Quote:
 Original post by hplus0603Regarding players per server, that depends on your degree of security, how much simulation you do, how interactive it is, and all that. ...If you do secure server-authenticated simulation at high simulation rates where each tool, attachment, vehicle, weapon, etc is a fully simulated object, and fully replicated to neighboring servers and clients, and have expensive rules for who can see what, when, and where, then you can cut that in ten.

Of course, only fools and the military (HLA is a strange acronym; those 3 letters are actually short for "painfully slow and too "H"igh-"L"evel to "A"ctually mean anything" :P :P) try to make all those thigns fully simulated objects without shortcutting :).

##### Share on other sites
Quote:
 Original post by graveyard fillaShadowbane, a commercial MMORPG which was designed from the start to support many players in a small area (fighting each other no less) could not handle this. in fact, when i was playing Shadowbane, it could not even handle 1/10th of that many players. even with 200 players the game would lag horribly and be in slow motion. theres really not much you can do about something like this.

Nah, that's just crap server design. The SB people had some great game- and client- developers but no real great server people IIRC.

Anyway, I vaguely remember one of them saying they'd fixed that 6+ months ago. I don't play it, so I can't confirm/deny. I do know that they have been worrying about other problems in the last three quarters, so I guess they got that fixed.

##### Share on other sites
A game like EverQuest runs about 100-200 players per piece of hardware (at least at the time of shipping -- when a 1 GHz CPU was the top of the line). They have maybe 50 zones in the game. They could put very heavily loaded zones onto their own machines, but put sveral zones that seldom have that many people all on a single machine to save on hardware cost. Thus, an initial EverQuest "server" (really, a world instance, or "shard") would be run by something like 30 machines.

What redmilamber and I are saying is that, with the right networking structure and game design, and today's PC hardware, you can probably put 2,000 players on a single piece of hardware -- but it depends a lot on what you want to do with the game, and thus how that informs the networking and simulation infrastructure you need on the server side. If you want all of everything, at high simulation rates, you won't be able to do 2,000 entities on a single machine, for much of the same reasons that you don't have 2,000 simultaneous active NPCs in a single-player game of Half-Life.

The easiest way to load balance between physical servers within a single shard is to require the user to "zone" between areas of the game, and put different zones on different machines -- the EQ model. The harder way is to let the user walk freely, and use shadow/ghost objects to hand off simulation responsibility between different physical servers. Asheron's Call did this, and many modern systems (including ours) does this. Our web site is somewhat low on details, but take a look at Forterra Systems for a few more details.

##### Share on other sites
Quote:
 Original post by Anonymous PosterHow many units did your previous games sell?...So. You mean "none", yes?

Actually, I dont know. We do budget games at present which means we get a fixed fee for license of our game. I dont care if it sells 1000 or 10.000 pieces. I only care for the license fee which at present is enough for us.

True, itd a BIG jump IF we started development of MMOG (though a 10 times feature-less than competitors have).

Quote:
 Original post by Anonymous PosterSure, it's doable. But, in reality, not unless you have one or more MMOG experts on the team or you have some top 0.001% people in your team (hey, it's possible; just very unlikely).I don't know your situation, I don't know what mega-genius you have, I'm just pointing out my experiences based on having worked with a lot of people in your position and they all went bankrupt, wound up the company, or simply gave up. And these were the ones with probably the best chances of success, carefully selected from hundreds.

It is very unlikely (aka impossible) to find some MMOG expert in my area. So from this point of view its impossible.

But it was also said to be impossible to create a game by only one programmer and one artist (in spare time after daily job) and yet we did it twice and are finishing our third 3D game. Of course MMOG cant be compared with Single-player budget games, theyre probably 100 times tougher to finish. Still, I could try. And for this Im gathering as much info as possible as to whether Ill try or not.

Quote:
 Original post by Anonymous PosterWell, obviously I'd say the one I know and use and helped write is "the best", although I'd also say that BigWorld's one is the only other good one (all the rest are moderate to mediocre (butterfly, zona) or just crap (being kind, I'll simply mention no names - it's pretty obvious who the useless ones are when you look at their tech docs).The other question is: how many of them can you afford? IIRC all the decent ones cost mahola...
Could you please be more specific and give me a link where pricing information could be found ? It probably is outside my range. But maybe I could buy it on leasing. Otherwise I wont know. Or just tell me the range in US$. Quote: Original post by Anonymous Poster Quote:  Questions : 1. Do you think that its impossible for one programmer to be adding new content and fixing bugs ? Possible, yes. I've seen it done. However, before you get to 2k people you absolutely MUST have dedicated "content" programmers and "bug-fixing" programmers. Otherwise what happens is one of those two jobs doesn't get done properly...it's jsut too hard, mentally, for one person to handle the workload without getting so depressed they get no work done at all, or simply ignore one aspect. If youre talking about development phase, thats natural to have these two programmers have separate roles. Or were you talking about launch phase ? Quote: Original post by Anonymous Poster Quote:  2. How many PCs are required for one server ? I heard stories of 50 PCs per server. Whats that mean ? We've been through this: in the above conversation, PC == server. What you're thikning about are clusters, and I wish everyone would call them that, because that's what they are, they are NOT servers. A cluster is a group of servers joined together to *simulate* a single server, but it is not (should not be) called "a server". Unfortuantely a significant nubmer of people insisit on muddying the waters :(. Thanks for clarifying. Now it makes more sense. Actually thats what I was thinking about originally, but since I dont know much about networks from inner side, I thought thats a separate issue to clusters. Quote:  Original post by Anonymous PosterBy the way - does anyone think it's worth me writing an article on this stuff? I could do a followup to "How to build an MMOG" (I think it's in the FAQ, or else google for it as a Stratics article) which actually contains practical numbers (like the staffing estimates), actual links, resources, etc. A couple of people (including two publishers) asked me to write a *book* on this topic, but I simply don't have *that* much time at the moment... A book is a mammoth task. Article is achievable more easily. It would be much appreciated. #### Share this post ##### Link to post ##### Share on other sites Quote:  Original post by hplus0603A game like EverQuest runs about 100-200 players per piece of hardware (at least at the time of shipping -- when a 1 GHz CPU was the top of the line). They have maybe 50 zones in the game. They could put very heavily loaded zones onto their own machines, but put sveral zones that seldom have that many people all on a single machine to save on hardware cost. Thus, an initial EverQuest "server" (really, a world instance, or "shard") would be run by something like 30 machines. Hm, I thought the main reason of slow-down (or low number of players handled) isnt the CPU but bandwidth. Or am I missing something here ? Quote:  Original post by hplus0603What redmilamber and I are saying is that, with the right networking structure and game design, and today's PC hardware, you can probably put 2,000 players on a single piece of hardware -- but it depends a lot on what you want to do with the game, and thus how that informs the networking and simulation infrastructure you need on the server side. If you want all of everything, at high simulation rates, you won't be able to do 2,000 entities on a single machine, for much of the same reasons that you don't have 2,000 simultaneous active NPCs in a single-player game of Half-Life. Itd be probably cheaper to have 2 different machines each handling ~1000 players. Besides, reliability&stability is very important here. So its better to have more hardware always. Because if one HW breaks, still at least half of people could play. But pissing off ALL customers is a Great NO-NO. But if your motherboard/CPU burns, bad luck anyway (and this happens a lot, at least to me in past-and no overclocking was present). Quote:  Original post by hplus0603The easiest way to load balance between physical servers within a single shard is to require the user to "zone" between areas of the game, and put different zones on different machines -- the EQ model. The harder way is to let the user walk freely, and use shadow/ghost objects to hand off simulation responsibility between different physical servers. Asheron's Call did this, and many modern systems (including ours) does this. Our web site is somewhat low on details, but take a look at Forterra Systems for a few more details. We definitely dont plan to let player go freely. If other commercial MMOGs get away with this, we wont be an exception complicating things for ourselves unnecesarilly. Though situation might be different in 2007. Still, itd be low-cost alternative to high-profile MMOGs. #### Share this post ##### Link to post ##### Share on other sites "They have maybe 50 zones in the game...." Everquest currently has 220+ zones in the game. Each zone is a separate instance running on a server. Some of the lower populated/vacant zones do run along side of other instances on the same physical machine. Some of these zones are instanced so that more than one of them can be running at the same time (a feature of the Lost Dungeons of Norrath expansion). These instances are created and destroyed on demand by players. Get 3-72 (size varies depending on if its for a group or a raid) of your buddies together and talk to an NPC and the dungeon instance is started up. Once you're done the instance is shutdown and destroyed. EQ also has 50 servers/shards that each run their own set of 220+ zones, so thats about ~11,000 zones running at any given time. Here's a list of zones =) Abysmal Sea Acrylia Caverns Ak'Anon Akheva Ruins Barindu - The Hanging Gardens Befallen Beholders Maze (Gorge of King Xorbb) Black Burrow Burning Woods Butcherblock Mountains Cabilis East Cabilis West Catacombs of Dranik Cazic Thule Charasis (Howling Stones) Chardok Chardok, Halls of Betrayal (Chardok B) Citadel of Anguish City of Mist Cobalt Scar Crushbone Crypt of Dalnir Crypt of Nadox Crystal Caverns Dagnors Cauldron Dawnshroud Peaks Deepest Guk Doomfire, the Burning Lands Dragon Necropolis Dranik's Hollows Dranik's Scar Dreadlands Drunder, Fortress of Zek (Plane of Tactics) Dulak's Harbor East Commonlands Eastern Karana Eastern Wastes Echo Caverns Emerald Jungle Erudin Erudin Palace Eruds Crossing Eryslai, the Kingdom of Wind Everfrost Peaks Feerrott Felwithe Ferubi - Forgotten Temple of Taelosia Field of Bone Firiona Vie Freeport East Freeport North Freeport West Frontier Mountains Fungus Grove Great Divide Greater Faydark Grieg's End Grimling Forest Gukta, Outpost of Marr Gulf of Gunthak Halas Halls of Honor Harbingers' Spire Hate's Fury Highpass Hold Highpass Keep Hollowshade Moor Iceclad Ocean Icewell Keep Ikkinz - Antechamber of Destruction Inktu'ta, the Unmasked Chapel Innothule Swamp Jaggedpine Forest Kael Drakkel Kaesora Kaladim North Kaladim South Karnor's Castle Katta Castellum Kedge Keep Kerafyrm's Lair (Sleeper's Tomb) Kerra Island Kithicor Forest Kod'Taz - Broken Trial Grounds Kurn's Tower Lair of Terris Thule Lake of Ill Omen Lake Rathetear Lavastorm Mountains Lesser Faydark Lower Guk Marauder's Mire Marus Seru Mines of Gloomingdeep Mines of Nurga Miragul's Menagerie Mistmoore Castle Mistmoore Catacombs Misty Thicket Mons Letalis Muramite Proving Grounds Nagafen's Lair (Solusek B) Najena Natimbi - The Broken Shores Nedaria's Landing Nektulos Forest Neriak 3rd Gate Neriak Commons Neriak Foreign Quarter Netherbian Lair Nobles' Causeway Northern Desert of Ro Northern Karana Oasis of Marr Ocean of Tears Oggok Old Sebilis Paineel Paludal Caverns Permafrost Plane of Disease Plane of Fear Plane of Growth Plane of Hate Plane of Innovation Plane of Justice Plane of Knowledge Plane of Mischief Plane of Nightmare Plane of Sky Plane of Storms Plane of Time A Plane of Time B Plane of Torment Plane of Tranquility Plane of Valor Qeynos Aqueducts Qeynos Hills Qeynos North Qeynos South Qinimi - Court of Nihilia Qvic - Prayer Grounds of Calling Ragrax, Stronghold of the Twelve Rathe Mountains Reef of Coirnav Riftseekers' Sanctum Rivervale Riwwi - Coliseum of Games Ruins of Lxanvom (Crypt of Decay) Ruins of Old Paineel (The Hole) Rujarkian Hills Runnyeye Sanctus Seru Sewers of Dranik Sewers of Nihilia - Emanating Crematory Sewers of Nihilia - Lair of Trapped Ones Sewers of Nihilia - Pool of Sludge Sewers of Nihilia - Purifying Plant Shadeweaver's Thicket Shadow Haven Shadowrest Shar Vahl Siren's Grotto Skyfire Mountains Skyshrine Solusek's Eye (Solusek A) Southern Desert of Ro Southern Karana Splitpaw Lair (Infected Paw) Ssraeshza Temple Steamfont Stonebrunt Mountains Surefall Glades Swamp of No Hope Tacvi, Seat of the Slaver Takish-Hiz Temple of Droga Temple of Marr Temple of Solusek Ro Temple of Veeshan The Arena The Bazaar The Bloodfields The Caverns of Exile (Solusek C) The Deep The Forgotten Halls The Grey The Maiden's Eye The Nexus The Overthere The Ruined City of Dranik The Scarlet Desert The Tenebrous Mountains The Twilight Sea The Umbral Plains Thurgadin Timorous Deep Tipt - The Treacherous Crags Torden, the Bastion of Thunder Torgiran Mines Tower of Frozen Shadow Tower of Solusek Ro Toxxulia Forest Trakanon's Teeth Txevu - Lair of the Elites Unrest Upper Guk Uqua - the Ocean God Chantry Veeshan's Peak Vegarlson, the Earthen Badlands Veksar Velketor's Labyrinth Vex Thal Vxed - The Crumbling Caverns Wakening Land Wall of Slaughter Warrens Warsliks Woods West Commonlands Western Karana Western Wastes Yxtta - Pulpit of Exiles -=[ Megahertz ]=- #### Share this post ##### Link to post ##### Share on other sites Quote:  VladR:Because if one HW breaks, still at least half of people could play. You need to figure out how you scale to multiple machines. For reasons of locality, you find that splitting by geographic area is usually the best performing. So, in this case, if one machine goes out, those parts of your world that are served by that machine, will be inaccessible. Unless you do like City of Heroes and instance the common world zones, too, based on load. If you want something that's existing, and in production, there is the engine behind Ryzom: NeL. I'm not terribly impressed technically, but it's open source, has shipped one title, and you can start developing content right now. Quote:  redmilamberDoes anyone think it's worth me writing an article on this stuff? I think that would be great. The more understanding there is for the range of options, how they're influenced by product design, and what they mean to your technology investment, the better. Quote:  MegaherzEverquest currently has 220+ zones in the game. I was talking about back when they started in 2000. I agree they have more zones now, and they probably are running it on more modern hardware than they did five years ago, too :-) #### Share this post ##### Link to post ##### Share on other sites Quote:  Original post by hplus0603The easiest way to load balance between physical servers within a single shard is to require the user to "zone" between areas of the game, and put different zones on different machines -- the EQ model. The harder way is to let the user walk freely, and use shadow/ghost objects to hand off simulation responsibility between different physical servers. Asheron's Call did this, and many modern systems (including ours) does this. Our web site is somewhat low on details, but take a look at Forterra Systems for a few more details. Or there's the grexengine way, which is a lot more efficient than ghosting (AC and most others of that ilk have massive synchronization overhead keeping the ghosts in synch). Maybe you use similar tricks to avoid the synch overhead, but still call it ghosting? In particular, there are some real nasty worst-case scenarios for ghosting/synch problems that I believe I've seen in action on several of the 50k+ MMORPGs causing massive (measurable in fractions of a minute or more!) lag. #### Share this post ##### Link to post ##### Share on other sites Quote:  Original post by VladRIt is very unlikely (aka impossible) to find some MMOG expert in my area. So from this point of view its impossible. But it was also said to be impossible to create a game by only one programmer and one artist (in spare time after daily job) and yet we did it twice LOL. Yeah, but *I* never believed that statement for even a minute, and I've always mocked the people who said it. Quote: Quote:  Original post by Anonymous PosterThe other question is: how many of them can you afford? IIRC all the decent ones cost mahola... Could you please be more specific and give me a link where pricing information could be found ? It probably is outside my range. But maybe I could buy it on leasing. Otherwise I wont know. Or just tell me the range in US$.

You'd have to ask them yourself, the last pricing figures I have for each are sufficiently out of date that I wouldn't post them publically - it's not fair to get it wrong if they've changed significantly in the last 12 months.

Most are open to negotiation though, depending upon the level of support and number of features you need. I can only think of two (relatively minor) that had fixed pricing plans...

Quote:
 If youre talking about development phase, thats natural to have these two programmers have separate roles. Or were you talking about launch phase ?

Forever. Especially post-launch.

Quote:
 A book is a mammoth task. Article is achievable more easily. It would be much appreciated.

Given the time-cost even for an article I might not manage it. The last article I wrote was proposed to a paper mag 11 months ago, planned to be published by April, and in the end I didn't finish writing it until November! (and it was only a couple of thousand words!!). Just very very busy...

As a friend said once "People who can do something spend all their time doing it, the rest simply write about it". :P

##### Share on other sites
"I was talking about back when they started in 2000."

I had a feeling you were, but I threw the numbers out there anyway.

Let me also add a few more things. The most populated EQ zone is the Bazaar which allows for people to set themselves up as traders however they must stay logged in to sell their wares. This zone typically has 300+ people in it and I've seen as high as 500ish.

One optimization that was made to support this high volume of users in the zone was that if a player isn't moving they're not sending updates to the server. I'd be willing to bet that if everybody started moving at once, it would crash the zone.

Another factor is just how densely packed people are in a zone. Some raid zones in EQ would bog down significantly if over 120 people were in the same general area, which was common when more than 1 guild was raiding there. However were you to split these guilds up and have one on one end of the zone and one on the other, things would improve a bit.

-=[ Megahertz ]=-

##### Share on other sites
Quote:
 Or there's the grexengine way, which is a lot more efficient than ghosting (AC and most others of that ilk have massive synchronization overhead keeping the ghosts in synch). Maybe you use similar tricks to avoid the synch overhead, but still call it ghosting?In particular, there are some real nasty worst-case scenarios for ghosting/synch problems that I believe I've seen in action on several of the 50k+ MMORPGs causing massive (measurable in fractions of a minute or more!) lag.

Our system runs real newtonian physics for every object, at 30 steps per second, using 4th-order runge-kutta integration -- it's a "distributed simulation," not an "RPG". The networking protocol is tightly integrated with the physics simulation. This means that if anything actually fell behind, it plain wouldn't work. This in turn means we have to limit the absolute total density of objects in any one area, or step down to a simler model when the servers detect overload.

Because the system is designed to let you play over a 28.8 kbps modem, the networking consumption of a single object is miniscule.

The benefits we get include server-authoritative, zero-perceived-latency, low-bandwidth, high-fidelity, secure physics. We can guarantee that any two objects that the simulation says should be able to interact, CAN interact with function-call latency (i e, nanoseconds). The cost is a finite upper limit on population density. Other systems that don't limit density, instead scale back the simulation when things get too complex, or they disallow interaction between objects, neither of which is the right choice for our platform's needs.

I wrote a white paper on how it all works, which I wish I could post here, but marketing wants an NDA to release it :-(

##### Share on other sites
If I get you right from that brief description, you've gone down the "full deterministic simulation, everywhere"? Which doesn't in and of itself explain how you avoid the problems of handover (biggest problem is the latency of a safe handover process without corrupting data or breaking the causal ordering of events in the handover zone?) unless your agent actions are pretty limited, so no chance of servers getting into a battle over what *exactly* the ND parts of hte system (players) have done in the last few seconds?

Or...there's always the AO approach: simply FREEZE each client for 3-60 seconds (!) when they move through a zone boundary and refuse to accept input in that time. Um. Ahem. Well, I guess it *works* in that it prevents corruption and keeps the handover deterministic, but it sucks in the game >:(.

Or have I completely misunderstood you ? :)

##### Share on other sites
You are correct: We are fully deterministic everywhere, and we have very fast (and fully predictable) hand-over. It's a hard problem to solve -- hence, we spent a lot of time on it, and charge money for our solution. Our platform solves a lot of other problems too -- it's a fully functional, modern, virtual world / distributed simulation toolkit, complete with rather long list of features deleted.

I think I can predict your next question, and will attempt to answer: There are, of course, cases where we are not 100% deterministic on all clients, such as under certain packet loss conditions, and when low-latency interaction results in your forward-predicted shadow copy of yourself being wrong. In the case of interaction with latency lower than one-way transmission latency, our solution makes sure that the actor (e g you shooting) sees the right thing; everyone spectating sees the same thing, and the victim (e g me getting shot) getting a correction which we design hiding mechanisms for depending on the actions. We find the ability to trade off latency versus accuracy for the "victim" being very useful, for example.

##### Share on other sites
To the original poster. You might find these links handy. They are written by the creator of Eternal Lands MMORPG and have some nice hints of what can go wrong and what really paid off etc:

A Beginner's Guide to Creating a MMORPG
Eternal Lands' MMORPG Postmortem: Mistakes and Lessons, Part I
Eternal Lands' MMORPG Postmortem: Mistakes and Lessons, Part II

##### Share on other sites
Quote:
 Original post by BorundinTo the original poster. You might find these links handy. They are written by the creator of Eternal Lands MMORPG and have some nice hints of what can go wrong and what really paid off etc:A Beginner's Guide to Creating a MMORPGEternal Lands' MMORPG Postmortem: Mistakes and Lessons, Part IEternal Lands' MMORPG Postmortem: Mistakes and Lessons, Part II

Nah, they suck. That first link, basically "I don't really know what I'm talking about, but I kind of had a few ideas, and some of them might be sort of the right kind of thing if you're lucky, but I haven't worked out which ones yet".

IMHO that first article would have been better never to have been written. It's like the X-Wing one: an article by someone who knows very little aimed at telling "other poeple who know very little" how to do it. You see the problem? Blind leading the blind...

##### Share on other sites
2nd and 3rd links are good, though. They are much more honest about how naive and ignorant the author is, hence much more useful.

##### Share on other sites
The thing that stands out from those links is this collection of important points:

1) you're going to get it wrong initially, so swallow your pride and promise that everything will probably change at some point
2) listening and reacting is the basic cycle of development, and absolutely necessary
3) that's especially true when it comes to things that detract from your ability to grow your subscriber base
4) some changes for the greater good are necessary, even if people complain, so you have to have the guts to make them

• 24
• 11
• 17
• 11
• 13