About multiplayer game testing

Started by
3 comments, last by Davi Doro 7 years, 11 months ago

Hello friends,

I have a nearly finished 1v1 turn-based strategy game (nearly meaning God knows when) that I would like to test. I have a match-making server that can run in any machine I want. I've already tested everything by myself, playing against myself running the server in a machine that I've rented at ops.mnx.io and everything works. I'm considering making an open beta for testing, let people play against each other, then I'll gather both feedback on the gameplay and the multiplayer. If I take that route, what things should I keep in mind? More specifically to the multiplayer, is there a way to predict the specifications of the machine I need to run a server at to be able to successfully handle let's say 50 games of chess happening at the same time? Is renting a machine hourly for running game servers a common practice?

I just don't want to publicly announce an open beta of the game and give people the wrong impression because things suck in some way, so for someone who is giving its first steps in the game industry as an indie, every advice is much appreciated!

Thanks!

Advertisement

Hello

For transparency. I work for a company called Multiplay and we are the world leaders in Game Server management and Hosting.

The only way to really know how many games you can run on a single virtual or bare metal server is to run a play test on your game server while monitoring the amount of resources needed for single game session. You can then calculate the total amount of available resource available to you in terms of Ghz. So 4 x 2.4Ghz cores would provide a total of 9.6Ghz. Best practise would be to run at a maximum of 80% utilisation. So if a single game used 0.5Ghz to run, for example, you could take 80% of 9.6Ghz and divide by 0.5Ghz per game server to give you the number of games per server.

I often advise anyone building a multi-player game to develop and test their game servers on the same infrastructure that you plan to use in production. It is amazing how many people are surprised when they setup the production environment only to find it does not work due to some strange configuration change on the server.

Picking a hosting company or in this case a Cloud Hosting company is very difficult. you need to be thinking of what you will do in the future. MNX that you linked to above is not a company I am familiar with so I cannot comment on them but the things that you want to consider are.

Locations - When you game launches how many locations around the world do you need. This is often dependant on how sensitive you game is to latency. For a turn based game like Chess it could be that a single location serving everyone in the world will be fine. If it is a FPS game you may need over 10 locations around the world in order to minimise the latency. Will your hosting company be able to support this?

Service - There are lots of different types of hosting companies. Will the hosting company give you the right level of support you need. If you have a 24 x 7 team to support your server OS system you might want to go with a cheaper provider who does not provide OS support. A managed hosting company might be more expensive but if something happens at 2am on a Sunday night will they help you get your game back online.

Uptime SLAs - different hosts have a wide verity of SLAs. If you just have a single location and the success of the game depends on the game servers you want to make sure that you pick one with the best SLA possible. If your going to have several locations and you are not making much money from the game you can get away with a lower hosting SLA.

Terms - If you rent a server by the hour you will be paying a premium for having that flexibility. If you rent a server for a longer period the cheaper it will get. If you just want to do some testing for a short period a hour to hour cloud server is perfect.

Price - Hosting and Cloud companies are all fighting for market share at the moment so prices are very low with very little difference between the best and the worst. So it is worth having the right one rather than the cheapest one.

Alternatively you could just go to a company that specialises in game server hosting and management who can do all of this for you. ;)

I hope that is helpful?

Run a load test.

Hopefully you didn't make the frequent beginner mistake of building a system that cannot run headless or that has no structured communications pattern. If you didn't that's understandable because it is a frequent beginner mistake, but it is a still a mistake.

If you have implemented it well, you can script behavior of running games without graphics, or replay games through recorded communications, or implement "monkey on a keyboard" scripts, or otherwise simulate a large number of players. Then you fire up a server and slowly add script runners. Start with five concurrent, then 10, then 20, then 50, then 100, then 150, then 200, then 300, then 500, until suddenly you hit a point where performance becomes unacceptable.

Usually it looks like a cliff, one moment the system is burdened but still running reasonably fast, and the next moment every client is waiting seconds for responses.

If you don't have that kind of design, then you'll have one step of getting a bunch of virtual machines on Amazon or somewhere else, firing all of them up and pointing them at your server and watching for it die a slow-performance death.

Is renting a machine hourly for running game servers a common practice?


Yes and no. For companies that are paying salaries of people, the cost of having some hardware always available for testing is small by comparison.
For indies who are self-funding, yes, renting hardware by the hour on Amazon or something similar is not a bad choice, as long as you understand the Amazon platform, and can live with virtualization.
Twitch FPS games may be somewhat sensitive to virtualization jitter, depending on specifics. It sounds like your game wouldn't be, so that's good.

One you're "deployed" you will typically have the machines available 24/7 for players, and if you are planning some marketing or other growth strategies, you'll want some extra machines ready to meet unexpected load.
For a game with FarmVille-style growth (very, very rare) you will have to automate the process of automatically inserting new capacity to the system, and you'd better be on a provider with sufficient spare capacity to soak up all your new users.
Smaller cloud/infrastructure companies typically have an upper limit to how much you can provision in one go, and typically have larger granules of payment than "by the hour" -- many charge by the month.
Similarly, the time to spin up a new server on the largest services is measured in seconds-to-few-minutes, whereas the time to spin up a new server on smaller places may be up to several hours.

So, it sounds like you're in the beginning of this journey. I agree with frob: If you can generate synthetic load/players, then that's great!
Apply that to your existing host, and see how far you get. Run a profiler on the server while you do this, to find what the bottlenecks are, which will likely show you some easy improvement opportunities.
This may be in database access, or in CPU processing, or in memory leaks, or in insufficient caching, or in one of a myriad other places -- you won't know until you measure!
Once that's all done, you should be able to run a beta for friends-and-family-and-friends-of-friends to get validation at small player counts.
After that, feel free to open it to the greater Internet. The biggest challenge at that point is getting anyone to actually care about your game, instead of the 100,000 other games that are out there, trying to get people to care for them :-(
enum Bool { True, False, FileNotFound };

MatthewMorris thank you for your detailed answer. Unfortunately, unless your company accepts ramen as a payment option, my college sponsorship does not cover game server hosting expenses. But since we talking business here, what are the chances that me and other companies sign a deal that stablishes you gain a percent of the game profit? They say its the next Hearthstone (they = my mom).

frob yes I think I can make a stress test of the server by my own, it would take 2 weeks or so to automate things but it's doable, so I guess I'll add that to my to do list.

hplus0603 yes, I'll make sure everyone and their mother care when the game is out. It is the best game of the universe after all. Now if you excuse me, I'll go to the marketing forum ask for help.

Thanks y'all. I'm smarter now.

This topic is closed to new replies.

Advertisement