Online Games Under Construction: Run Your Beta Right

Jonathan Hanna (Sr. Producer, John Galt Games), Richard Weil (Director of Community Relations, Cartoon Network), April Burba

Treat your beta as Live

When you run your Beta, don’t just pretend that the game has gone live, act as if the game has gone live, because it has! Sure you’re not opening up to hundreds of thousands of players but you’re still servicing the game to several hundred, maybe even several thousands of people and they will demand a service equivalent of the kind they get for a regular online game. So obviously you won’t have everything that the final live service will have, and your players will understand that the game is not complete, but do not use this as an excuse to hide behind when things go south for some reason and people get upset. For instance if a batch of players get their accounts wiped without being notified because a database was reset to get new metrics data, they won’t be like “oh, well – it’s a beta so no big deal.” No, they will be very upset because they won’t know it was a wipe, will think it’s a service issue and will believe that perceived quality of the product will carry over into the final version and stop playing. So treat it as a live service and keep all your players informed. Many may sign up for a beta without a full understanding of what it means to participate in such a program, so make sure that every player involved understands what will be expected of them during the process.

Players are not testing during a beta, they are previewing. During their play the development team is pulling down data and using that to clean up the experience and squash any bugs. You should not have to make your players directly participate in the testing process by having them submit bug or crash reports themselves – they should be left alone to play the game and have fun while the game logs any errors or crashes to send back to the dev team. If they come across a problem with the game play, you’ll no doubt have forums set up where they can voice those complaints.

Goals for a beta

Several things you want to strive for when carrying out a beta:

• Building Community – keep players involved in the beta process by giving them reasons to play the game and work with each other to explore areas and uncover any problems.
• Setting a tone – take this time to work out with your dev team how to approach communicating with the community. Obviously you don’t want the dev team becoming too chummy with the community, as that can lead the players to believe they carry great weight in design decisions. On the other hand you don’t want the team being too stand-offish and making the community think their ideas are not being heard. Strike the balance.
• Building a contact list – get to know valuable players in the community; they may be leveraged later on as managers for various contents and community events.
• Define internal processes – work with your dev team on how to handle community issues (issues players have with each other) and how to support the players that have issues with the game.
• Marketing/PR – use the players to hype up your game. Get them hooked on cool features that they can use to tease any of their friends not involved in the beta. Find out what works and what doesn’t and apply that to your overall campaign.
• Testing new ideas – this is the best time to implement new ideas or features, when the impact is relatively small and maneageble. Beta considerations

Expect churn. Churn is the term used to describe the turn-over of players in your beta. You will have beta players become discouraged for many reasons, including them just not liking the game. Whatever the reason is, let them go and don’t try to force them to stay – take their reason for leaving into account.

Necessary evils will have to be performed over the course of a beta, and you should both make players aware of this at the start, and make them have to deal with it as few times as possible. These include character resets, server downtime, stress tests and lots of game updates.

Don’t muddle your player’s minds with over-complex NDA’s. If you must make players sign NDA’s so they can’t blab about the game, be sure to lighten-up on the legalese and make it clear what players can and cannot talk about in relation to the game during the beta process.

Getting and keeping players

Let players be able to form guilds or groups in the game from the start. Better yet, make it available for people to register for the beta as a group.

Once a player registers for the beta, let them come back and update their registration (if they have not yet been selected) in case they made any changes to their hardware configurations. Obviously make sure the dev team is notified of these changes in case they’re looking for more people running, for instance, a certain graphics card model.

Allow players to turn around and invite friends into the beta. This doesn’t have to be at the very start, but at some point when you’re ready to expand the beta, instead of selecting new users, let the current player base bring in new users. This is a nice “reward” for beta players, and makes them want to play more now that they have more friends in the game as well.

Run lots of events for focus-testing purposes. For example, if there’s a new dungeon that you want stress tested, challenge players to beat said dungeon in a certain amount of time to win a prize. Contests like these are good ways to energize the community while taking in valuable data. City of Heroes ran a costume contest to test their costume design feature. Be careful though that you don’t make such things appear to be common-place, or beta players may feel like the service has been devalued once the switch is made to live and less contests are offered (unless you plan to keep up regular events into live).

Beta lessons

Your beta is a huge focus test for your game, be sure to record as much data as you can!

Use aggressive surveys, give out rewards for completing them, and let them be completed both in-game and out, as well as anonymously.

Talk to churned-out players and learn why they’re leaving.

Metrics! (for more on metrics, see this talk)

Speak up on issues you receive from the community and address them. Ask yourselves: why will your product fail?

Using your beta for sales

As mentioned before, don’t let the NDA ruin the player’s ability to talk up your game to their friends and even the world at large. Be specific but at the same time clear enough that player’s understand what they can and cannot say, rather than be confused and choose not to speak for fear of legal action. If NDA breaks do happen, combat them in a positive way.

Run journals, Q&A, etc., that keep the public in the loop about what’s going on with the beta. Don’t forget about your non-beta audience that is still waiting to hop into the game.

Have a link to the beta signup page placed in publicity pieces.

Coverage by Drew Sikora

Page 2