• Advertisement
Sign in to follow this  

MMORPG Servers question

This topic is 4775 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi, We are exploring the possibility to start the development of MMORPG later this year. Our plans are to serve max. 2.000 people ($5 per month). Maybe that doesn`t count much like MMORPG, but it surely isn`t _just_ common multiplayer. Besides I don`t think we could compete feature-wise with HUGE projects where 150 people work on this. Still, somehow I think that 2.000 is reasonable number that is possible to reach, if at least our graphics shall be great and common gameplay features are implemented. It would be our fourth or fifth game project already so we`d have enough experience for basic game development. Besides we should have our graphics renderer quite advanced by that time (if everything goes well). The questions: 1. How many players can usually safely connect to 1 server ? I.e. How many servers/PCs would we need to serve those ~2.000 people ? 2. Are there any commercial libraries (with support) for Multiplayer component of MMORPG (codebase) ? 3. How many people would we need to make this all run 24/7 (i.e. support,shards,network specialists) ?

Share this post


Link to post
Share on other sites
Advertisement
Guest Anonymous Poster
Quote:
Original post by VladR
It would be our fourth or fifth game project already so we`d have enough experience for basic game development. Besides we should have our graphics renderer quite advanced by that time (if everything goes well).


How many units did your previous games sell? IF they were merely hobby projects that didn't sell then you still have too little experience to write, maintain, and run an MMOG.

Quote:

1. How many players can usually safely connect to 1 server ?


Thousands. Any less and you've got a crap networking layuer / arch.

Quote:

I.e. How many servers/PCs would we need to serve those ~2.000 people ?


Depends entirely on how complex your game is; if it's a real-time MMOFPS with lots of time-critical gameplay and large amounts of data pinging around then you'll might need as many as a server per couple of hundred players. However, IMHO that only happens if your architecture is crap, your network programmers are crap, and your game-design is crap. It takes considerable incompetence to be unable to manage 500 players per server, and in most games you should be able to manage 1k.

If it's a turn-based combat game with multiple seconds per turn and turn-locked actions then you should be able to manage many thousands per server.

To put this into perspective, one of the biggest (by player numbers) online MMORPG's with 3D graphical world, semi-realtime combat (the norm, but NOT quake-style) handles 3,000 players per physical server, and their logic-execution + network-pipeline are both less efficient than they could be (although not by much, yet I think I could squeeze quite a few more players on there)

Some here will probably tell you you can't get anywhere near that.

Quote:

2. Are there any commercial libraries (with support) for Multiplayer component of MMORPG (codebase) ?


Plenty. There's even a website devoted to listing them (you'll have to google for it, something like "MMOG technology List" or something like that), although its' very basic info only.

Shameless plug: http://grexengine.com (although the site is extremely plain. Just email sales @ grexengine.com and you can get the marketing info etc emailed to you)

Quote:

3. How many people would we need to make this all run 24/7 (i.e. support,shards,network specialists) ?


For 2k players teh rule of thumb is you need approx 3-7 full-time staff (developers + support + technicians). I've seen 15k players supported, commercially, by a total staff fewer than 5, but that's exceptional (if you've got an excellent project manager on-board and a cunning game-desing, you could manage this too, but I wouldn't recommend it).

i.e. you need a minimum (always) of one CSR, one dev (adding new features), and one maintenance (bug-fixer). For 10k players, you're aiming for a total staff of approx 5-15, for 50k players you want 8-20, all depending upon your game etc.

Share this post


Link to post
Share on other sites
Thanks for feedback, here are my answers:

Quote:
How many units did your previous games sell? IF they were merely hobby projects that didn't sell then you still have too little experience to write, maintain, and run an MMOG.
I`d definitely agree with Maintaining and running MMOG, though I disagree with Writing MMOG. As I said, it`s gonna be less feature-rich than e.g. Everquest2 (at least 10 times less, that is). This is accomplishable in 2 years with small team with graphics renderer (terrain/characters/animation/special effects/Collision Detection) already completed.

Quote:

Thousands. Any less and you've got a crap networking layuer / arch.

Sounds good considering we`d be happy with 2.000 subscribers.

Quote:

Quote:

I.e. How many servers/PCs would we need to serve those ~2.000 people ?


Depends entirely on how complex your game is; if it's a real-time MMOFPS with lots of time-critical gameplay and large amounts of data pinging around then you'll might need as many as a server per couple of hundred players. However, IMHO that only happens if your architecture is crap, your network programmers are crap, and your game-design is crap. It takes considerable incompetence to be unable to manage 500 players per server, and in most games you should be able to manage 1k.
Well, it`s gonna be RPG, so movement isn`t going to be that fast as in FPS, still I can imagine that submitting information about 2.000 players each second (or so) is going to be hard to achieve, but compression is my friend (or so I guess).
And 1000 people sounds good. That would mean I`d need just 2 servers. That`s accomplishable.

Quote:

Quote:

2. Are there any commercial libraries (with support) for Multiplayer component of MMORPG (codebase) ?


Plenty. There's even a website devoted to listing them (you'll have to google for it, something like "MMOG technology List" or something like that), although its' very basic info only.

Shameless plug: http://grexengine.com (although the site is extremely plain. Just email sales @ grexengine.com and you can get the marketing info etc emailed to you)
Thanks for links, I`ll check those out (some of them I already checked). Naturally I googled already, I just hoped someone will provide a real-world feedback for the library they work with (e.g. like you).

Quote:

Quote:

3. How many people would we need to make this all run 24/7 (i.e. support,shards,network specialists) ?


For 2k players teh rule of thumb is you need approx 3-7 full-time staff (developers + support + technicians). I've seen 15k players supported, commercially, by a total staff fewer than 5, but that's exceptional (if you've got an excellent project manager on-board and a cunning game-desing, you could manage this too, but I wouldn't recommend it).
Sounds reasonable, actually that`s what I found out elsewhere so far too. So it would be doable on $10k monthly revenue.

Questions :
1. Do you think that it`s impossible for one programmer to be adding new content and fixing bugs ?
2. How many PCs are required for one server ? I heard stories of 50 PCs per server. What`s that mean ?



Share this post


Link to post
Share on other sites
you might be able to save on bandwith my doing something like what Shadowbane did, by using the mouse for movement. i thought it was a pretty good idea. Shadowbane required to use as little bandwith as possible, since it was sort of action based, and encouraged huge battles, Player vs Player battles where one guild would storm anothers castle.

a little OT but, the Shadowbane game design was amazing, but in reality it just didnt work. whenever a huge battle would take place, you and 50 people go to storm a castle, all hell (lag) broke lose. the whole castle storming thing was going on in _VERY_ slow motion.

however, that was years ago when the game first came out. id be very interested in knowing if they ever solved this problem.

sorry for slightly de-railing your thread.

Share this post


Link to post
Share on other sites
I don't know what country and state you're in. In California, junior programmers cost at least $5k/month, and people who are experienced go for much more than that. Customer service might be cheaper, but there's still rent, and power, and cooling, and bandwidth, and hardware leasing... $10,000/month wouldn't work at all around here for a company size 3-7.

Regarding players per server, that depends on your degree of security, how much simulation you do, how interactive it is, and all that. If all you're doing is bouncing a few packets a second between players that are close to each other, you can easily go to 2,000 on a single piece of modern hardware. After all, most routers are just regular computers, and do many more packets than that. If you do secure server-authenticated simulation at high simulation rates where each tool, attachment, vehicle, weapon, etc is a fully simulated object, and fully replicated to neighboring servers and clients, and have expensive rules for who can see what, when, and where, then you can cut that in ten. If you're a typical RPG, you can design towards the high end of that. If you're a military simulation, you find yourself at the lower end (but then, your customer might have money to spend on hardware :-)

Quote:
For 10k players, you're aiming for a total staff of approx 5-15, for 50k players you want 8-20, all depending upon your game etc.


I can second these numbers.

There's also a question of man-hours versus hardware cost. If your people cost you $10k/month, and you can buy a $1k server to solve a problem that would take a person one week to solve, then buying the server makes more sense. I imagine that if your team is based in some country with cheap labor, the equation would look very different -- although what the size of the market would be is then less clear (trying to support the US market from abroad is a tough nut to crack).

Share this post


Link to post
Share on other sites
Quote:
Original post by hplus0603
I don't know what country and state you're in. In California, junior programmers cost at least $5k/month, and people who are experienced go for much more than that. Customer service might be cheaper, but there's still rent, and power, and cooling, and bandwidth, and hardware leasing... $10,000/month wouldn't work at all around here for a company size 3-7.
I`m from Slovakia as can be seen from the header of the thread. This means I can get junior programmers in my area for about $300 per month. Still, if I offer $500 per month, they`ll be lining up in front of my door (I`m in region with 23% unemployment rate (highest in whole country) where average salary is about $300). True, add 37% to that because of taxes&social insurance, but still it`s far from $5k as in California. Office holding 10 people costs around $300 per month.
Although, I`d have to add about 20% to that because of inflation and the fact that it wouldn`t start sooner that 2006/2007 IF we shall start the development later this year.

Quote:
Original post by hplus0603Regarding players per server, that depends on your degree of security, how much simulation you do, how interactive it is, and all that. If all you're doing is bouncing a few packets a second between players that are close to each other, you can easily go to 2,000 on a single piece of modern hardware.
Well, and what about if all 2000 players are close to each other (not necessarily on same screen), but close enough to send packets about all 2.000 players ? How do I scale this into more servers ?

Quote:
Original post by hplus0603After all, most routers are just regular computers, and do many more packets than that. If you do secure server-authenticated simulation at high simulation rates where each tool, attachment, vehicle, weapon, etc is a fully simulated object, and fully replicated to neighboring servers and clients, and have expensive rules for who can see what, when, and where, then you can cut that in ten. If you're a typical RPG, you can design towards the high end of that. If you're a military simulation, you find yourself at the lower end (but then, your customer might have money to spend on hardware :-)
I guess only time will tell. But I needed to know the worst scenario as to how many severs do I need to buy and setup and maintain.


Quote:
Original post by hplus0603There's also a question of man-hours versus hardware cost. If your people cost you $10k/month, and you can buy a $1k server to solve a problem that would take a person one week to solve, then buying the server makes more sense. I imagine that if your team is based in some country with cheap labor, the equation would look very different
Could you elaborate more about what do you mean by the example of $1k server ? Like it has software/configuration that prevents some type of crashes/bugs/issues that are otherwise handled only manually ?

Quote:
Original post by hplus0603although what the size of the market would be is then less clear
Did you mean to say that we`re just trying to adress home-country market ? LOL, I doubt I`d find 5 people here, this is Slovakia with most of people having dial-up being paid by time (i.e. the more you`re online the more you pay - pricing per minute). Those $5 they`d pay for 3 hrs of connection during day-time. Although there is pretty strong base of people having broadband (like me).
Naturally, since this is MMOG, we`ll try to adress the world market where I think it is possible to find 2.000 subscribers paying $5 each month. I could be wrong though of course. And of course it would have to be new people, since our game would hardly have features that would cause EQ2 players to quit and start here.

Quote:
Original post by hplus0603(trying to support the US market from abroad is a tough nut to crack).
Well, I think that we could easily cover 18 hours per day of customer service, though 24/7 would be harder to achieve although most probably necessary. What do you mean by supporting US market from abroad ? Answering phone calls ? No problem with language here (except for salary these people require).

I still would like to know what do they usually mean when they say that 1 server is run by 50 PCs. Does it mean I`ll have to buy 100 PCs for two servers if I want each server to be able to handle 1.000 players ?

Share this post


Link to post
Share on other sites
Quote:
Original post by VladR
Well, and what about if all 2000 players are close to each other (not necessarily on same screen), but close enough to send packets about all 2.000 players ? How do I scale this into more servers ?


my guess is simply the game will lag horribly. i have never even heard of (and definetly never played) a MMORPG which could support 2000 players in the same area. Shadowbane, a commercial MMORPG which was designed from the start to support many players in a small area (fighting each other no less) could not handle this. in fact, when i was playing Shadowbane, it could not even handle 1/10th of that many players. even with 200 players the game would lag horribly and be in slow motion. theres really not much you can do about something like this.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Quote:
Original post by VladR
Thanks for feedback, here are my answers:

Quote:
How many units did your previous games sell?

...


So. You mean "none", yes?

Sure, it's doable. But, in reality, not unless you have one or more MMOG experts on the team or you have some top 0.001% people in your team (hey, it's possible; just very unlikely).

I don't know your situation, I don't know what mega-genius you have, I'm just pointing out my experiences based on having worked with a lot of people in your position and they all went bankrupt, wound up the company, or simply gave up. And these were the ones with probably the best chances of success, carefully selected from hundreds.

Quote:

Naturally I googled already, I just hoped someone will provide a real-world feedback for the library they work with (e.g. like you).


Well, obviously I'd say the one I know and use and helped write is "the best", although I'd also say that BigWorld's one is the only other good one (all the rest are moderate to mediocre (butterfly, zona) or just crap (being kind, I'll simply mention no names - it's pretty obvious who the useless ones are when you look at their tech docs).

The other question is: how many of them can you afford? IIRC all the decent ones cost mahola...

Quote:

Questions :
1. Do you think that it`s impossible for one programmer to be adding new content and fixing bugs ?


Possible, yes. I've seen it done. However, before you get to 2k people you absolutely MUST have dedicated "content" programmers and "bug-fixing" programmers. Otherwise what happens is one of those two jobs doesn't get done properly...it's jsut too hard, mentally, for one person to handle the workload without getting so depressed they get no work done at all, or simply ignore one aspect.

Quote:

2. How many PCs are required for one server ? I heard stories of 50 PCs per server. What`s that mean ?


We've been through this: in the above conversation, PC == server.

What you're thikning about are clusters, and I wish everyone would call them that, because that's what they are, they are NOT servers. A cluster is a group of servers joined together to *simulate* a single server, but it is not (should not be) called "a server". Unfortuantely a significant nubmer of people insisit on muddying the waters :(.

By the way - does anyone think it's worth me writing an article on this stuff? I could do a followup to "How to build an MMOG" (I think it's in the FAQ, or else google for it as a Stratics article) which actually contains practical numbers (like the staffing estimates), actual links, resources, etc. A couple of people (including two publishers) asked me to write a *book* on this topic, but I simply don't have *that* much time at the moment...

redmilamber

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Quote:
Original post by hplus0603
Regarding players per server, that depends on your degree of security, how much simulation you do, how interactive it is, and all that.
...
If you do secure server-authenticated simulation at high simulation rates where each tool, attachment, vehicle, weapon, etc is a fully simulated object, and fully replicated to neighboring servers and clients, and have expensive rules for who can see what, when, and where, then you can cut that in ten.


Of course, only fools and the military (HLA is a strange acronym; those 3 letters are actually short for "painfully slow and too "H"igh-"L"evel to "A"ctually mean anything" :P :P) try to make all those thigns fully simulated objects without shortcutting :).

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Quote:
Original post by graveyard filla
Shadowbane, a commercial MMORPG which was designed from the start to support many players in a small area (fighting each other no less) could not handle this. in fact, when i was playing Shadowbane, it could not even handle 1/10th of that many players. even with 200 players the game would lag horribly and be in slow motion. theres really not much you can do about something like this.


Nah, that's just crap server design. The SB people had some great game- and client- developers but no real great server people IIRC.

Anyway, I vaguely remember one of them saying they'd fixed that 6+ months ago. I don't play it, so I can't confirm/deny. I do know that they have been worrying about other problems in the last three quarters, so I guess they got that fixed.

Share this post


Link to post
Share on other sites
A game like EverQuest runs about 100-200 players per piece of hardware (at least at the time of shipping -- when a 1 GHz CPU was the top of the line). They have maybe 50 zones in the game. They could put very heavily loaded zones onto their own machines, but put sveral zones that seldom have that many people all on a single machine to save on hardware cost. Thus, an initial EverQuest "server" (really, a world instance, or "shard") would be run by something like 30 machines.

What redmilamber and I are saying is that, with the right networking structure and game design, and today's PC hardware, you can probably put 2,000 players on a single piece of hardware -- but it depends a lot on what you want to do with the game, and thus how that informs the networking and simulation infrastructure you need on the server side. If you want all of everything, at high simulation rates, you won't be able to do 2,000 entities on a single machine, for much of the same reasons that you don't have 2,000 simultaneous active NPCs in a single-player game of Half-Life.

The easiest way to load balance between physical servers within a single shard is to require the user to "zone" between areas of the game, and put different zones on different machines -- the EQ model. The harder way is to let the user walk freely, and use shadow/ghost objects to hand off simulation responsibility between different physical servers. Asheron's Call did this, and many modern systems (including ours) does this. Our web site is somewhat low on details, but take a look at Forterra Systems for a few more details.

Share this post


Link to post
Share on other sites
Quote:
Original post by Anonymous Poster
How many units did your previous games sell?
...
So. You mean "none", yes?

Actually, I don`t know. We do budget games at present which means we get a fixed fee for license of our game. I don`t care if it sells 1000 or 10.000 pieces. I only care for the license fee which at present is enough for us.

True, it`d a BIG jump IF we started development of MMOG (though a 10 times feature-less than competitors have).

Quote:
Original post by Anonymous Poster
Sure, it's doable. But, in reality, not unless you have one or more MMOG experts on the team or you have some top 0.001% people in your team (hey, it's possible; just very unlikely).

I don't know your situation, I don't know what mega-genius you have, I'm just pointing out my experiences based on having worked with a lot of people in your position and they all went bankrupt, wound up the company, or simply gave up. And these were the ones with probably the best chances of success, carefully selected from hundreds.

It is very unlikely (aka impossible) to find some MMOG expert in my area. So from this point of view it`s impossible.

But it was also said to be impossible to create a game by only one programmer and one artist (in spare time after daily job) and yet we did it twice and are finishing our third 3D game. Of course MMOG can`t be compared with Single-player budget games, they`re probably 100 times tougher to finish. Still, I could try. And for this I`m gathering as much info as possible as to whether I`ll try or not.


Quote:
Original post by Anonymous Poster
Well, obviously I'd say the one I know and use and helped write is "the best", although I'd also say that BigWorld's one is the only other good one (all the rest are moderate to mediocre (butterfly, zona) or just crap (being kind, I'll simply mention no names - it's pretty obvious who the useless ones are when you look at their tech docs).

The other question is: how many of them can you afford? IIRC all the decent ones cost mahola...
Could you please be more specific and give me a link where pricing information could be found ? It probably is outside my range. But maybe I could buy it on leasing. Otherwise I won`t know. Or just tell me the range in US$.

Quote:
Original post by Anonymous Poster
Quote:
Questions :
1. Do you think that it`s impossible for one programmer to be adding new content and fixing bugs ?


Possible, yes. I've seen it done. However, before you get to 2k people you absolutely MUST have dedicated "content" programmers and "bug-fixing" programmers. Otherwise what happens is one of those two jobs doesn't get done properly...it's jsut too hard, mentally, for one person to handle the workload without getting so depressed they get no work done at all, or simply ignore one aspect.
If you`re talking about development phase, that`s natural to have these two programmers have separate roles. Or were you talking about launch phase ?


Quote:
Original post by Anonymous Poster
Quote:

2. How many PCs are required for one server ? I heard stories of 50 PCs per server. What`s that mean ?


We've been through this: in the above conversation, PC == server.

What you're thikning about are clusters, and I wish everyone would call them that, because that's what they are, they are NOT servers. A cluster is a group of servers joined together to *simulate* a single server, but it is not (should not be) called "a server". Unfortuantely a significant nubmer of people insisit on muddying the waters :(.
Thanks for clarifying. Now it makes more sense. Actually that`s what I was thinking about originally, but since I don`t know much about networks from inner side, I thought that`s a separate issue to clusters.


Quote:
Original post by Anonymous Poster
By the way - does anyone think it's worth me writing an article on this stuff? I could do a followup to "How to build an MMOG" (I think it's in the FAQ, or else google for it as a Stratics article) which actually contains practical numbers (like the staffing estimates), actual links, resources, etc. A couple of people (including two publishers) asked me to write a *book* on this topic, but I simply don't have *that* much time at the moment...
A book is a mammoth task. Article is achievable more easily. It would be much appreciated.

Share this post


Link to post
Share on other sites
Quote:
Original post by hplus0603A game like EverQuest runs about 100-200 players per piece of hardware (at least at the time of shipping -- when a 1 GHz CPU was the top of the line). They have maybe 50 zones in the game. They could put very heavily loaded zones onto their own machines, but put sveral zones that seldom have that many people all on a single machine to save on hardware cost. Thus, an initial EverQuest "server" (really, a world instance, or "shard") would be run by something like 30 machines.
Hm, I thought the main reason of slow-down (or low number of players handled) isn`t the CPU but bandwidth. Or am I missing something here ?

Quote:
Original post by hplus0603What redmilamber and I are saying is that, with the right networking structure and game design, and today's PC hardware, you can probably put 2,000 players on a single piece of hardware -- but it depends a lot on what you want to do with the game, and thus how that informs the networking and simulation infrastructure you need on the server side. If you want all of everything, at high simulation rates, you won't be able to do 2,000 entities on a single machine, for much of the same reasons that you don't have 2,000 simultaneous active NPCs in a single-player game of Half-Life.
It`d be probably cheaper to have 2 different machines each handling ~1000 players. Besides, reliability&stability is very important here. So it`s better to have more hardware always. Because if one HW breaks, still at least half of people could play. But pissing off ALL customers is a Great NO-NO. But if your motherboard/CPU burns, bad luck anyway (and this happens a lot, at least to me in past-and no overclocking was present).

Quote:
Original post by hplus0603The easiest way to load balance between physical servers within a single shard is to require the user to "zone" between areas of the game, and put different zones on different machines -- the EQ model. The harder way is to let the user walk freely, and use shadow/ghost objects to hand off simulation responsibility between different physical servers. Asheron's Call did this, and many modern systems (including ours) does this. Our web site is somewhat low on details, but take a look at Forterra Systems for a few more details.
We definitely don`t plan to let player go freely. If other commercial MMOGs get away with this, we won`t be an exception complicating things for ourselves unnecesarilly. Though situation might be different in 2007. Still, it`d be low-cost alternative to high-profile MMOGs.

Share this post


Link to post
Share on other sites
"They have maybe 50 zones in the game...."

Everquest currently has 220+ zones in the game. Each zone is a separate instance running on a server. Some of the lower populated/vacant zones do run along side of other instances on the same physical machine.

Some of these zones are instanced so that more than one of them can be running at the same time (a feature of the Lost Dungeons of Norrath expansion). These instances are created and destroyed on demand by players. Get 3-72 (size varies depending on if its for a group or a raid) of your buddies together and talk to an NPC and the dungeon instance is started up. Once you're done the instance is shutdown and destroyed.

EQ also has 50 servers/shards that each run their own set of 220+ zones, so thats about ~11,000 zones running at any given time.


Here's a list of zones =)


Abysmal Sea
Acrylia Caverns
Ak'Anon
Akheva Ruins
Barindu - The Hanging Gardens
Befallen
Beholders Maze (Gorge of King Xorbb)
Black Burrow
Burning Woods
Butcherblock Mountains
Cabilis East
Cabilis West
Catacombs of Dranik
Cazic Thule
Charasis (Howling Stones)
Chardok
Chardok, Halls of Betrayal (Chardok B)
Citadel of Anguish
City of Mist
Cobalt Scar
Crushbone
Crypt of Dalnir
Crypt of Nadox
Crystal Caverns
Dagnors Cauldron
Dawnshroud Peaks
Deepest Guk
Doomfire, the Burning Lands
Dragon Necropolis
Dranik's Hollows
Dranik's Scar
Dreadlands
Drunder, Fortress of Zek (Plane of Tactics)
Dulak's Harbor
East Commonlands
Eastern Karana
Eastern Wastes
Echo Caverns
Emerald Jungle
Erudin
Erudin Palace
Eruds Crossing
Eryslai, the Kingdom of Wind
Everfrost Peaks
Feerrott
Felwithe
Ferubi - Forgotten Temple of Taelosia
Field of Bone
Firiona Vie
Freeport East
Freeport North
Freeport West
Frontier Mountains
Fungus Grove
Great Divide
Greater Faydark
Grieg's End
Grimling Forest
Gukta, Outpost of Marr
Gulf of Gunthak
Halas
Halls of Honor
Harbingers' Spire
Hate's Fury
Highpass Hold
Highpass Keep
Hollowshade Moor
Iceclad Ocean
Icewell Keep
Ikkinz - Antechamber of Destruction
Inktu'ta, the Unmasked Chapel
Innothule Swamp
Jaggedpine Forest
Kael Drakkel
Kaesora
Kaladim North
Kaladim South
Karnor's Castle
Katta Castellum
Kedge Keep
Kerafyrm's Lair (Sleeper's Tomb)
Kerra Island
Kithicor Forest
Kod'Taz - Broken Trial Grounds
Kurn's Tower
Lair of Terris Thule
Lake of Ill Omen
Lake Rathetear
Lavastorm Mountains
Lesser Faydark
Lower Guk
Marauder's Mire
Marus Seru
Mines of Gloomingdeep
Mines of Nurga
Miragul's Menagerie
Mistmoore Castle
Mistmoore Catacombs
Misty Thicket
Mons Letalis
Muramite Proving Grounds
Nagafen's Lair (Solusek B)
Najena
Natimbi - The Broken Shores
Nedaria's Landing
Nektulos Forest
Neriak 3rd Gate
Neriak Commons
Neriak Foreign Quarter
Netherbian Lair
Nobles' Causeway
Northern Desert of Ro
Northern Karana
Oasis of Marr
Ocean of Tears
Oggok
Old Sebilis
Paineel
Paludal Caverns
Permafrost
Plane of Disease
Plane of Fear
Plane of Growth
Plane of Hate
Plane of Innovation
Plane of Justice
Plane of Knowledge
Plane of Mischief
Plane of Nightmare
Plane of Sky
Plane of Storms
Plane of Time A
Plane of Time B
Plane of Torment
Plane of Tranquility
Plane of Valor
Qeynos Aqueducts
Qeynos Hills
Qeynos North
Qeynos South
Qinimi - Court of Nihilia
Qvic - Prayer Grounds of Calling
Ragrax, Stronghold of the Twelve
Rathe Mountains
Reef of Coirnav
Riftseekers' Sanctum
Rivervale
Riwwi - Coliseum of Games
Ruins of Lxanvom (Crypt of Decay)
Ruins of Old Paineel (The Hole)
Rujarkian Hills
Runnyeye
Sanctus Seru
Sewers of Dranik
Sewers of Nihilia - Emanating Crematory
Sewers of Nihilia - Lair of Trapped Ones
Sewers of Nihilia - Pool of Sludge
Sewers of Nihilia - Purifying Plant
Shadeweaver's Thicket
Shadow Haven
Shadowrest
Shar Vahl
Siren's Grotto
Skyfire Mountains
Skyshrine
Solusek's Eye (Solusek A)
Southern Desert of Ro
Southern Karana
Splitpaw Lair (Infected Paw)
Ssraeshza Temple
Steamfont
Stonebrunt Mountains
Surefall Glades
Swamp of No Hope
Tacvi, Seat of the Slaver
Takish-Hiz
Temple of Droga
Temple of Marr
Temple of Solusek Ro
Temple of Veeshan
The Arena
The Bazaar
The Bloodfields
The Caverns of Exile (Solusek C)
The Deep
The Forgotten Halls
The Grey
The Maiden's Eye
The Nexus
The Overthere
The Ruined City of Dranik
The Scarlet Desert
The Tenebrous Mountains
The Twilight Sea
The Umbral Plains
Thurgadin
Timorous Deep
Tipt - The Treacherous Crags
Torden, the Bastion of Thunder
Torgiran Mines
Tower of Frozen Shadow
Tower of Solusek Ro
Toxxulia Forest
Trakanon's Teeth
Txevu - Lair of the Elites
Unrest
Upper Guk
Uqua - the Ocean God Chantry
Veeshan's Peak
Vegarlson, the Earthen Badlands
Veksar
Velketor's Labyrinth
Vex Thal
Vxed - The Crumbling Caverns
Wakening Land
Wall of Slaughter
Warrens
Warsliks Woods
West Commonlands
Western Karana
Western Wastes
Yxtta - Pulpit of Exiles



-=[ Megahertz ]=-

Share this post


Link to post
Share on other sites
Quote:
VladR:
Because if one HW breaks, still at least half of people could play.


You need to figure out how you scale to multiple machines. For reasons of locality, you find that splitting by geographic area is usually the best performing. So, in this case, if one machine goes out, those parts of your world that are served by that machine, will be inaccessible. Unless you do like City of Heroes and instance the common world zones, too, based on load.

If you want something that's existing, and in production, there is the engine behind Ryzom: NeL. I'm not terribly impressed technically, but it's open source, has shipped one title, and you can start developing content right now.

Quote:
redmilamber
Does anyone think it's worth me writing an article on this stuff?


I think that would be great. The more understanding there is for the range of options, how they're influenced by product design, and what they mean to your technology investment, the better.

Quote:
Megaherz
Everquest currently has 220+ zones in the game.


I was talking about back when they started in 2000. I agree they have more zones now, and they probably are running it on more modern hardware than they did five years ago, too :-)

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Quote:
Original post by hplus0603
The easiest way to load balance between physical servers within a single shard is to require the user to "zone" between areas of the game, and put different zones on different machines -- the EQ model. The harder way is to let the user walk freely, and use shadow/ghost objects to hand off simulation responsibility between different physical servers. Asheron's Call did this, and many modern systems (including ours) does this. Our web site is somewhat low on details, but take a look at Forterra Systems for a few more details.


Or there's the grexengine way, which is a lot more efficient than ghosting (AC and most others of that ilk have massive synchronization overhead keeping the ghosts in synch). Maybe you use similar tricks to avoid the synch overhead, but still call it ghosting?

In particular, there are some real nasty worst-case scenarios for ghosting/synch problems that I believe I've seen in action on several of the 50k+ MMORPGs causing massive (measurable in fractions of a minute or more!) lag.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Quote:
Original post by VladR
It is very unlikely (aka impossible) to find some MMOG expert in my area. So from this point of view it`s impossible.

But it was also said to be impossible to create a game by only one programmer and one artist (in spare time after daily job) and yet we did it twice


LOL. Yeah, but *I* never believed that statement for even a minute, and I've always mocked the people who said it.

Quote:

Quote:
Original post by Anonymous Poster

The other question is: how many of them can you afford? IIRC all the decent ones cost mahola...
Could you please be more specific and give me a link where pricing information could be found ? It probably is outside my range. But maybe I could buy it on leasing. Otherwise I won`t know. Or just tell me the range in US$.


You'd have to ask them yourself, the last pricing figures I have for each are sufficiently out of date that I wouldn't post them publically - it's not fair to get it wrong if they've changed significantly in the last 12 months.

Most are open to negotiation though, depending upon the level of support and number of features you need. I can only think of two (relatively minor) that had fixed pricing plans...

Quote:

If you`re talking about development phase, that`s natural to have these two programmers have separate roles. Or were you talking about launch phase ?



Forever. Especially post-launch.

Quote:

A book is a mammoth task. Article is achievable more easily. It would be much appreciated.


Given the time-cost even for an article I might not manage it. The last article I wrote was proposed to a paper mag 11 months ago, planned to be published by April, and in the end I didn't finish writing it until November! (and it was only a couple of thousand words!!). Just very very busy...

As a friend said once "People who can do something spend all their time doing it, the rest simply write about it". :P

Share this post


Link to post
Share on other sites
"I was talking about back when they started in 2000."


I had a feeling you were, but I threw the numbers out there anyway.

Let me also add a few more things. The most populated EQ zone is the Bazaar which allows for people to set themselves up as traders however they must stay logged in to sell their wares. This zone typically has 300+ people in it and I've seen as high as 500ish.

One optimization that was made to support this high volume of users in the zone was that if a player isn't moving they're not sending updates to the server. I'd be willing to bet that if everybody started moving at once, it would crash the zone.

Another factor is just how densely packed people are in a zone. Some raid zones in EQ would bog down significantly if over 120 people were in the same general area, which was common when more than 1 guild was raiding there. However were you to split these guilds up and have one on one end of the zone and one on the other, things would improve a bit.


-=[ Megahertz ]=-

Share this post


Link to post
Share on other sites
Quote:

Or there's the grexengine way, which is a lot more efficient than ghosting (AC and most others of that ilk have massive synchronization overhead keeping the ghosts in synch). Maybe you use similar tricks to avoid the synch overhead, but still call it ghosting?

In particular, there are some real nasty worst-case scenarios for ghosting/synch problems that I believe I've seen in action on several of the 50k+ MMORPGs causing massive (measurable in fractions of a minute or more!) lag.


Our system runs real newtonian physics for every object, at 30 steps per second, using 4th-order runge-kutta integration -- it's a "distributed simulation," not an "RPG". The networking protocol is tightly integrated with the physics simulation. This means that if anything actually fell behind, it plain wouldn't work. This in turn means we have to limit the absolute total density of objects in any one area, or step down to a simler model when the servers detect overload.

Because the system is designed to let you play over a 28.8 kbps modem, the networking consumption of a single object is miniscule.

The benefits we get include server-authoritative, zero-perceived-latency, low-bandwidth, high-fidelity, secure physics. We can guarantee that any two objects that the simulation says should be able to interact, CAN interact with function-call latency (i e, nanoseconds). The cost is a finite upper limit on population density. Other systems that don't limit density, instead scale back the simulation when things get too complex, or they disallow interaction between objects, neither of which is the right choice for our platform's needs.

I wrote a white paper on how it all works, which I wish I could post here, but marketing wants an NDA to release it :-(

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
If I get you right from that brief description, you've gone down the "full deterministic simulation, everywhere"? Which doesn't in and of itself explain how you avoid the problems of handover (biggest problem is the latency of a safe handover process without corrupting data or breaking the causal ordering of events in the handover zone?) unless your agent actions are pretty limited, so no chance of servers getting into a battle over what *exactly* the ND parts of hte system (players) have done in the last few seconds?

Or...there's always the AO approach: simply FREEZE each client for 3-60 seconds (!) when they move through a zone boundary and refuse to accept input in that time. Um. Ahem. Well, I guess it *works* in that it prevents corruption and keeps the handover deterministic, but it sucks in the game >:(.

Or have I completely misunderstood you ? :)

Share this post


Link to post
Share on other sites
You are correct: We are fully deterministic everywhere, and we have very fast (and fully predictable) hand-over. It's a hard problem to solve -- hence, we spent a lot of time on it, and charge money for our solution. Our platform solves a lot of other problems too -- it's a fully functional, modern, virtual world / distributed simulation toolkit, complete with rather long list of features deleted.

I think I can predict your next question, and will attempt to answer: There are, of course, cases where we are not 100% deterministic on all clients, such as under certain packet loss conditions, and when low-latency interaction results in your forward-predicted shadow copy of yourself being wrong. In the case of interaction with latency lower than one-way transmission latency, our solution makes sure that the actor (e g you shooting) sees the right thing; everyone spectating sees the same thing, and the victim (e g me getting shot) getting a correction which we design hiding mechanisms for depending on the actions. We find the ability to trade off latency versus accuracy for the "victim" being very useful, for example.

Share this post


Link to post
Share on other sites
To the original poster. You might find these links handy. They are written by the creator of Eternal Lands MMORPG and have some nice hints of what can go wrong and what really paid off etc:

A Beginner's Guide to Creating a MMORPG
Eternal Lands' MMORPG Postmortem: Mistakes and Lessons, Part I
Eternal Lands' MMORPG Postmortem: Mistakes and Lessons, Part II

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Quote:
Original post by Borundin
To the original poster. You might find these links handy. They are written by the creator of Eternal Lands MMORPG and have some nice hints of what can go wrong and what really paid off etc:

A Beginner's Guide to Creating a MMORPG
Eternal Lands' MMORPG Postmortem: Mistakes and Lessons, Part I
Eternal Lands' MMORPG Postmortem: Mistakes and Lessons, Part II


Nah, they suck. That first link, basically "I don't really know what I'm talking about, but I kind of had a few ideas, and some of them might be sort of the right kind of thing if you're lucky, but I haven't worked out which ones yet".

IMHO that first article would have been better never to have been written. It's like the X-Wing one: an article by someone who knows very little aimed at telling "other poeple who know very little" how to do it. You see the problem? Blind leading the blind...

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
2nd and 3rd links are good, though. They are much more honest about how naive and ignorant the author is, hence much more useful.

Share this post


Link to post
Share on other sites
The thing that stands out from those links is this collection of important points:

1) you're going to get it wrong initially, so swallow your pride and promise that everything will probably change at some point
2) listening and reacting is the basic cycle of development, and absolutely necessary
3) that's especially true when it comes to things that detract from your ability to grow your subscriber base
4) some changes for the greater good are necessary, even if people complain, so you have to have the guts to make them

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement