MMORPG Servers
Author
Does anyone have a good source of information on servers for MMORPG's. I am looking to understand what the hardware, OS and primary applications look like and will take any direction that might be provided.
TIA!
Welcome to GameDev.Net.
I'd start off with Searching GameDev.net.
Some good information can be found from the threads and articles linked... such as this one:
MMORPG - How many servers should I buy?
General consenus I believe is:
1) The optimal server hardware will vary widely depending on the game (MMO pong or FPS might be limited mainly by network bandwidth (tons of updates every second), an MMORTS more likely by CPU (tons of unit interactions to be resolved), a highly immersive MMORPG might be limited by RAM (tons of content), etc.
2) You won't be able to determine the "optimal" hardware until you can actually run your near-finished software on it. You can make educated guesses, but they'll be just that - guesses.
3) You shouldn't even worry about this until you've got near-finished software anyways - why fork out money now when it'll be cheaper later? Obviously, you'll have to fork out money eventually so you can get people playing - but the longer you put it off, the cheaper it'll become.
4) You can use any OS that supports networking, by doing all resource organizing/updating and syncronization at the software level. There are some resources that may help ease this task, normally used in Beowulf clusters (hint: there's another search term). A few linux kernel patches come to mind. HURD is supposed to have some of these features in the future, don't know the current status of that project.
I'd start off with Searching GameDev.net.
Some good information can be found from the threads and articles linked... such as this one:
MMORPG - How many servers should I buy?
General consenus I believe is:
1) The optimal server hardware will vary widely depending on the game (MMO pong or FPS might be limited mainly by network bandwidth (tons of updates every second), an MMORTS more likely by CPU (tons of unit interactions to be resolved), a highly immersive MMORPG might be limited by RAM (tons of content), etc.
2) You won't be able to determine the "optimal" hardware until you can actually run your near-finished software on it. You can make educated guesses, but they'll be just that - guesses.
3) You shouldn't even worry about this until you've got near-finished software anyways - why fork out money now when it'll be cheaper later? Obviously, you'll have to fork out money eventually so you can get people playing - but the longer you put it off, the cheaper it'll become.
4) You can use any OS that supports networking, by doing all resource organizing/updating and syncronization at the software level. There are some resources that may help ease this task, normally used in Beowulf clusters (hint: there's another search term). A few linux kernel patches come to mind. HURD is supposed to have some of these features in the future, don't know the current status of that project.
Author
MaulingMonkey
Thanks for the reply. I have bounced around on searches and have a lot more to go. Also, I checked the thread you referenced and it was quite informative. I have a couple of specific questions:
I have seen little reference to multithreading, 64-bit Xeon/Opteron and clustering. It would seem to me that these kinds of technologies would be ideal for serving an MMO game. Am I missing something or are these technologies priced out of the market?
Also, would you recommend checking the hosting sites to get a good idea of the config for a server, even if I don't have a firm number on users, yet? I guess I am just wondering if the likes of HOSTIK, EV1Servers, http://www.fastservers.net/, ServerBeach, ServerMatrix, 1 and 1 have what could be considered accurate configs and competitive pricing.
Thanks, again.
Thanks for the reply. I have bounced around on searches and have a lot more to go. Also, I checked the thread you referenced and it was quite informative. I have a couple of specific questions:
I have seen little reference to multithreading, 64-bit Xeon/Opteron and clustering. It would seem to me that these kinds of technologies would be ideal for serving an MMO game. Am I missing something or are these technologies priced out of the market?
Also, would you recommend checking the hosting sites to get a good idea of the config for a server, even if I don't have a firm number on users, yet? I guess I am just wondering if the likes of HOSTIK, EV1Servers, http://www.fastservers.net/, ServerBeach, ServerMatrix, 1 and 1 have what could be considered accurate configs and competitive pricing.
Thanks, again.
Quote:Original post by lyleis
I have seen little reference to multithreading, 64-bit Xeon/Opteron and clustering. It would seem to me that these kinds of technologies would be ideal for serving an MMO game. Am I missing something or are these technologies priced out of the market?
Areas multithreading applies:
1) Blocking I/O - common to all server applications using this model, covered seperately/by traditional models, nothing MMO-specific.
2) Taking advantage of multiple processors - only useful for multiprocessor systems. For those systems, traditional models will probably apply all the same. Nothing really MMO-specific.
Googling for information on this probably shouldn't include the term "MMO".
Areas 64-bit applies:
1) Extending directly addressable RAM usage. Only useful if you're truely limited by this. You're more likely to be limited by network throughput and CPU speed in most circumstances, I'd think.
2) Allowing for addressing clusterwide memory within a single range. Only useful if your OS is configured for this.
3) Faster data movement. This will speed the copy of large amounts of data between sections of RAM due to the increased size of registers.
1 & 2 can be worked around with using files, more than 4GB of which can be kept completely cached in memory if your motherboard supports it, and your OS supports segmentation in 32-bit mode (entirely possible with an x86, believe it or not). You could optionally map in/out of directly-addressable memory as needed.
Besides #2, none if this is particularly MMO specific. Even with #2, this applies to clustering technologies in general... it wouldn't be particularly unique to 64-bit.
Quote:Also, would you recommend checking the hosting sites to get a good idea of the config for a server, even if I don't have a firm number on users, yet?
I'd recommend only worrying about hosting when you want to enter beta-testing. Chances are in the early stages of development you'll want to be able to play around with server and networking configurations enough to make offsite hosting prohibitively time consuming until it's fairly finalized. Fact is, you shouldn't need more than 2 or 3 test servers during early testing, only as you head to stress testing and massive ammounts of users does that number go up. I say 2 or 3 only because that way you can test/debug problems arising from having to share the world representation with multiple servers.
Well, 2-3 test servers would be for a fairly big mmorpg. We got away with just one and only recently upgrade to a dual opteron ( yes we're taking advantage of the dual cpu ). We're a twitch space shooter kind of game so it's quite cpu intensive.
Quote:Am I missing something or are these technologies priced out of the market?
Most server shards use some kind of clustering (in the sense of machines that interchange data about the running world), although it's usually custom-written. The reason for this is likely that most commercial clustering solutions are not aimed at the low-latency types of interactions that games require.
Other kinds of clustering software is aimed at setting up and maintaining the machine set, rather than supporting the application as such; you can pick and choose there (but you SHOULD choose something -- building it on your own is pointless).
When it comes to hosting, once you're ready to go live, you'll likely want to set up a co-location hosting with some high-quality data center. That way, you can get the specific server configuration you need, and the specific networking requiremetns you need, as well as actual access to the machines (which is important!) For beta, you could likely go with one of the higher-quality self-managed server solutions, such as ServerBeach or 1-and-1.
However, for development, do it all in-house. You won't need to worry about leasing servers for quite some time!
Author
Great comments. Thanks for all the help. Keep em coming!
I think I understand that I needn't worry too much about the hardware until the game is complete (or, nearly). However, it becomes a "chicken and egg" proposition. Some choices of hardware will require special considerations in the code, etc. For example, I have a friend who works for SGI who touts their Altix as being superior to clusters (partitioning, scalability, better I/O, single system image, shared memory, true 64-bit, backplane). However, I don't know if that kind of system is completely out of the ballpark, pricewise, and might be overkill as an MMO server. But, surely, if I were chose a platform like that, I would have many considerations to manage as the server code is being written.
Does anyone know specifics on the servers of any of the big players, for me to use as a baseline? Or, for that matter, specifics on indies or smaller configurations, that might give me an idea of what has been tried and what is working.
Thanks to all for your help.
I think I understand that I needn't worry too much about the hardware until the game is complete (or, nearly). However, it becomes a "chicken and egg" proposition. Some choices of hardware will require special considerations in the code, etc. For example, I have a friend who works for SGI who touts their Altix as being superior to clusters (partitioning, scalability, better I/O, single system image, shared memory, true 64-bit, backplane). However, I don't know if that kind of system is completely out of the ballpark, pricewise, and might be overkill as an MMO server. But, surely, if I were chose a platform like that, I would have many considerations to manage as the server code is being written.
Does anyone know specifics on the servers of any of the big players, for me to use as a baseline? Or, for that matter, specifics on indies or smaller configurations, that might give me an idea of what has been tried and what is working.
Thanks to all for your help.
Quote:Original post by Dark Rain
Well, 2-3 test servers would be for a fairly big mmorpg. We got away with just one and only recently upgrade to a dual opteron ( yes we're taking advantage of the dual cpu ). We're a twitch space shooter kind of game so it's quite cpu intensive.
Or one that does fairly dynamic load balancing, I'd imagine. Although I suppose you can probably get away with doing most of the testing just running multiple processes of the server program on the same box, for some testing...
Ok, well I can give you my indie perspective on a commercial mmorpg. We're called starsonata. It's a space based mmorpg, with real time combats, meaning it's all about your flying skills. Items too, of course, but you get my point. Calculations have to be made for bullet/lasers collisions with the objects in the galaxy etc.
We've started with a P4 and it worked for a while but we're running apache and mysql on it too to lower our costs and at 800 galaxies, when apache or mysql started some background task that took a bit too much CPU, the game started lagging for players.
We've switched to a dual opetron 244 now and while we're still running mysql and apache on it, it's quite smooth and we can run 2800 galaxies without any measurable effect on the game speed.
To give an idea of how many objects have to be update in one loop, galaxies can range from having 0 to 5 suns in them, with multiple planets orbiting them and a LOT of asteroid in some system and a couple in oters. All of that has to be moved each frame and collision calculated. Then there's the AI, we have a lot of mobs running around the game world, it's about 30-40% of our CPU buget. I can't begin to guess how many thousands of AI are moving around but it's a lot and each one has to deal with inertia to calculate it's movement, aim, fire etc.
Player bases have to be updated too, progress on the production of player made items monitored, extraction of material from planets and planetary colonies update too for some bases etc.
In short, I couldn't believe at first all that was running on a P4 with no raid and a gig of ram. Heck, when I started they weren't very familiar with gcc and they weren't using the O flags ( 01, 02, 03 ) and it was compiled with the debug flag too. That was about 3-4 months after deployement of the live server. I'm still surprised at how inexpensive a world simulation is to run vs a 3d game client or a single player game. I'm guessing most programmer tend to do the same mistake as me and over estimate the ressource needed to run a mmorpg server.
Oh I mean sure, if you're WOW, EQ or EVE-Online you're going to need more ressources than that I'd say on average indie mmorpg don't get that much traffic at first. You need to start slow and upgrade as needed if you're on any sort of indie grade budget.
We've started with a P4 and it worked for a while but we're running apache and mysql on it too to lower our costs and at 800 galaxies, when apache or mysql started some background task that took a bit too much CPU, the game started lagging for players.
We've switched to a dual opetron 244 now and while we're still running mysql and apache on it, it's quite smooth and we can run 2800 galaxies without any measurable effect on the game speed.
To give an idea of how many objects have to be update in one loop, galaxies can range from having 0 to 5 suns in them, with multiple planets orbiting them and a LOT of asteroid in some system and a couple in oters. All of that has to be moved each frame and collision calculated. Then there's the AI, we have a lot of mobs running around the game world, it's about 30-40% of our CPU buget. I can't begin to guess how many thousands of AI are moving around but it's a lot and each one has to deal with inertia to calculate it's movement, aim, fire etc.
Player bases have to be updated too, progress on the production of player made items monitored, extraction of material from planets and planetary colonies update too for some bases etc.
In short, I couldn't believe at first all that was running on a P4 with no raid and a gig of ram. Heck, when I started they weren't very familiar with gcc and they weren't using the O flags ( 01, 02, 03 ) and it was compiled with the debug flag too. That was about 3-4 months after deployement of the live server. I'm still surprised at how inexpensive a world simulation is to run vs a 3d game client or a single player game. I'm guessing most programmer tend to do the same mistake as me and over estimate the ressource needed to run a mmorpg server.
Oh I mean sure, if you're WOW, EQ or EVE-Online you're going to need more ressources than that I'd say on average indie mmorpg don't get that much traffic at first. You need to start slow and upgrade as needed if you're on any sort of indie grade budget.
Quote:We've switched to a dual opetron 244 now and while we're still running mysql and apache on it, it's quite smooth and we can run 2800 galaxies without any measurable effect on the game speed.
Did you compare the price of just adding a second cheap P4 server to run the simulation on, and keep the apache/MySQL on the existing machine? It's been my experience that two simple machines often are cheaper than one complex machine, and typically scale at least as well, if not better.
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
