The purpose of this series is to do just that: to take an academic look at the MMO genre as a whole and see what we can learn about this growing genre and gaming as a whole as well. Backed by academia, through the The Guildhall @ SMU, and built on a foundation of in-depth interviews with the developers down in the trenches making these games, this article series is both an intellectual exercise and learning experience for all of us and an attempt to offer something back to the development community that is, unfortunately, in limited supply: a source of discourse on game design.
In the last article we examined the future of the genre in light of the high profile cancellations and failures that have appeared over the last year or two. Was the genre doomed? Was the success of the Ultimas, the Everquests, the Lineages, etc just a fluke? To find the answer I approached founding fathers Richard Garriott and Brad McQuaid. You can see the whole article here:
Our final analysis was, no, not at all. The MMO genre is simply following the same path that all genres do: an initial boom and then a period of stabilization. So with the MMORPG here to stay, what's next?
The world of the MMORPG is perhaps the highest-risk project in the industry. However it is also one of the most rewarding. Because of that, on paper, it would seem that it's a genre best left to the experts.
But that's not going to happen. Developers either entirely unproven or unproven in the genre are going to step up to the plate. And rightfully so. Nearly every huge MMO to date has come from a developer who was new to the genre. Ultima Online and Everquest were pioneers but more recently City of Heroes and World of Warcraft were all by companies that had never produced an MMOG.
So what do developers need to know if they hope to tread the genre's dangerous waters? Where are the Scyllas and Cyclops of their journeys?
In hope of answering this I ventured down to the offices of NCsoft, located in the hills outside Austin, Texas. NCsoft is in the unique position of being exclusive to the MMO genre and being both a developer and a publisher of third party titles.
There we held a roundtable to answer these very issues. The cast included some of the most well known and influential members of the genre:
Peter Freese, Core Technology Director
Jeremy Gaffney, Vice President of Product Development
Starr Long, Producer, Tabula Rasa
Valerie Massey, Online Community Coordinator, Auto Assault/Tabula Rasa
Paul Sage, Lead Designer, Tabula Rasa
Chris Strasz, Designer, Tabula Rasa
Richard Weil, Online Community Relations Manager
With the goal of providing insight into what MMOG developers need to know to survive in the next generation firmly in mind we discussed topics which touched on all aspects of the industry, beginning with publishers, traveling through development, into beta, and onward to release and beyond. The following chapters form an account of that journey and while they do not provide a road map for success - for indeed no such thing exists - they should provide valuable insight to any would be developer of MMORPG titles.
Chapter 1: What Are Generations?
"Generation" is a term that is often thrown around when speaking about the industry as a whole. MMORPGs are no different. Currently MMOs are in a transitional period. Are they in their second generation or third? What is a generation?
A generation of technology is, much like its etymological parent, defined by a period of growth more than a period of time. Unfortunately, in an industry trapped between hard numbers and hard marketing such a nebulous concept often leads more to debate than consensus. Nevertheless it is unavoidable. It is a term widely in use and firmly entrenched. Therefore we should do our best to define it.
We see two primary opinions: Jeremy Gaffney and Starr Long both agree that the term as a whole is poorly suited to the genre. There has been little truly revolutionary throughout the growth of the genre and so, they argue, is it truly worth considering anything a new generation? Is polishing and expanding upon already existing technologies and mechanics worthy of the title "generation". From a pure technical viewpoint it's a point that's hard to argue against.
There are two sides to this coin however. If we return to the root of the term we quickly realize that there are two ways one can look at the concept. If we use the First Person Shooter genre as an example we can see the growth of a genre practically defined by the latest ID and Epic engines. It's easy to define something as Doom, Quake, or Doom III era. From this viewpoint the first thought is entirely correct. Conversely, has the move from Everquest and Lineage to Everquest II and Lineage II done much to redefine the genre? From a purely technical perspective, absolutely not.
The other view is that a generation is more of a "systemic band". A period of time in which the community of developers creates a "generation" of titles followed by a new period of time where developers (either the same developers or others) create new titles which will, inevitably, form a reaction to the successes and failures of the games that came before them.
Richard Weil first brought up the term "systemic band" in this quote, "I think you can see some kind of systemic bands that people are referring to as generations. If you look at the first generation as people who sort of launched themselves off a cliff and my sense is that the second generation is people sort of polishing things they saw go wrong with the first generation. And I hope that with the third generation you'll see more of what we commonly call innovation."
Given that the term is not going away I prefer the second school of thought. It's semi-quantifiable and somewhat useful since we can see cause and effect within the model. That means that the first generation was those who "launched themselves off a cliff." Titles such as Meridian 59, Ultima Online, Everquest, Lineage, Anarchy Online, Asheron's Call and others and it marks a period of time roughly encompassing the late nineties.
The second would be the games who could watch the successes and failures of the first generation. It also features our first sequels. This would include most of the games released after the turn of the century. Dark Age of Camelot, Everquest II, Lineage II, City of Heroes, Shadowbane, World of Warcraft, etc
And this would leave the third generation as our future: Tabula Rasa, Vanguard, Dungeons & Dragons Online, Guild Wars, Imperator and many others. All of these are titles that will need to look carefully upon the successes and failures of both generations to succeed.
Of course this definition reflects only on the mass market titles. Only those titles that are Massively Multiplayer are really included because for years previous to any of these titles the MUD community was quietly refining the experience and laying the foundation that all of these games would be built on. And this is a very important addendum. But within these guidelines we have a workable division. Fluid, perhaps, but workable. And that in itself is an important step in researching the genre as it evolves.
Chapter 2: Gaffney's Four Rules of Success
Publishers are a fact of life with every game. The word "Massively" is a part of the MMOG title for a reason. These defining titles are big, huge games, both in terms of players and content. Developing these massive undertakings takes a commensurate amount of capitol and someone has to foot the bill. While not the cornerstone of a title they are the first step towards having a title see the light of day. They are the gatekeepers.
As the Vice President of Product Development at NCsoft Austin Jeremy Gaffney sees hundreds of game pitches a year. Of those only a very few are chosen to be NCsoft products. But how are those projects chosen? Now while there is no equation, no magic bullet that will tell you what to do, Gaffney outlines NCsoft's simple plan:
So there's management. There's art. Do you have good artists who can execute? Programming, you know it's a very complex technological task to do this. Design, because it's also very hard to quantify.
And also you're really looking for innovation which is really hard to judge when you're in the early stage of pitches. You know, is this going to work or not? Contrary to the popular take on publishers I think most publishers really are looking for innovation. Because most people are bright enough to know that clones just don't sell. I mean name me a successful clone; its pretty tough to think of one.
Really what we're looking for is fun. If you can show us a fun demo you can screw up everything else in your game as long as it's fun. That's a really good stepping point to actually be able to get a pitch. If you have a demo you bring in that the publisher is actually going to be playing in their off hours you win. Because there's no way that game's not being signed up."
The difficulty is switching between those extremes. It's a personnel and personality issue. Let's take a fictional lead artist as an example. On a team of five you're probably The Artist. Consistency, quality, and timeliness are all your responsibility and job. But what happens when you're now a team of 50? Maybe you have ten guys working under you now. You've gone, probably over the course of only a few months, from creating 100% of the art to 5% or less. It's still your responsibility but it's no longer your job. Each person copes with this in different ways. Some well and some not so well.
What do you do if you're the producer on that team and you know The Artist, who is probably also your friend, isn't going to adapt well? What do you do about the person who insists on doing it "the way it's always been" when new processes are put in for a reason?
The answers to these questions are beyond the scope of this article. Never-the-less they're questions that a developer has to consider. If you can step up and show a publisher a plan for your expansion you can help yourself greatly. Manage expectations. If The Artist knows from early on that he'll be a senior artist and not the lead artist it'll be easier for him when the time comes for someone to step up to that position. It's things like this that publishers are looking for - with good reason, as this is the sort of fundamental business element in projects like this that often gets overlooked and, in turn, causes significant pain to both the project and the personnel.
Rule 2: Do you have the skills? The first half of this is obvious. Video games push the boundaries of technology constantly. You need people who can handle truly advanced technologies quickly. MMO games are no different at the most basic level. However, what is worth considering is the ways they are different. You need network code that rivals financial databases. You need technology that allows you to get the best look and frame rate with fifty characters on screen all wearing different things. You need modular character designs. You need the tightest security possible. The list goes on. You may not have to be on the cutting edge of normal maps and real time lighting but the pitfalls are just as difficult and just as great.
Rule 3: Have you innovated or improved the genre in some way? Innovation is important. As Gaffney said, clones don't sell. This is, perhaps, even more important in the MMORPG genre as these games are inherently "sticky" in design. With offline games you just need to move the box and, hopefully, make the customer happy enough that you'll be able to move the next box as well. But with an online game you must plan to keep people playing for months and months if not years and years. Because this is a necessity you must not only provide a reason to play your game but a reason for people to give up the game they're already playing, all the while, attempting to bring new customers into the fold.
It's a tall order. Pretty graphics and slick features help, but in the end you must bring something new to the table to stand out.
Rule 4: Is it fun? This is the golden rule. Every developer knows it. Every player knows it. But it's also the easiest thing to lose perspective of. It's easy to tout features. These sorts of things are easy to quantify. So people, developers and players alike, will latch onto them. It is always easier for the mind to accept the concrete over the abstract. The developer must always keep in mind the Golden Rule.
There is a simple reason that the bullet points on boxes sell: because even if the consumer does not directly realize it the concepts they promote sound fun. That's it.
A concrete example would be Doom III. Normal mapping and real-time lighting sold copies. Not because players wanted to view the technological achievement, but because it looked good. And why did this matter to those customers? Because the better looking the game is the more fun it is to play.
Time and again Gaffney came back to this point. To re-quote him: "...you can screw up everything else in your game as long as it's fun."
Chapter 3: The Melding of Single Player and Multiplayer
The treatment of the MMORPG as a genre separate from other genres is a delicate balance and one that is becoming both increasingly difficult to justify and increasingly important to consider.
In the end massively multiplayer online games are just that. They are games. And while there are some inherent differences, as with any genre, many of the things that worked in other games will work in MMORPGs. As the genre grows it will diversify and you'll see sub-genres within it that will, by and large, tend to reflect the major genres of smaller scale games.
The crux is that there are only a few differences between the two types of games. However, thus far, those few differences have driven a rather large wedge between game styles. However, as the genre evolves we will see a further integration of what is successful traveling both ways as mechanics and techniques will be lifted from the single player world and transported to multiplayer and vice versa.
Chris Strasz brought up a solid point about developers as a whole:
Jeremy Gaffney explains the dilemma well in this quote:
If we shipped a game that had ten times that much content, say it only had a hundred hours of gameplay in it, we'd be reamed! Like there's no way that people would play. They'd blaze through it in two weeks and say "Why should I give you fifteen more bucks?"
And so the bar is just higher in terms of content between online and off. And it ought to be, it's not an unfair thing, we want their money every month while the single player games only want it once."
This is already happening in many of the games that have released in the last year or two. The cinematic experience is being drawn on by companies like Square-Enix, SOE, and Blizzard. World of Warcraft and Everquest 2 are good examples because they're both big worlds, like previous games, but they're also surrounded by a framework that attempts to aid the player in finding a direction from which to experience the world through the use of lore, storyline, and quests. It remains to be seen how successful they will be but their success, both short-term (which seems high at this point) and long term, will have to be evaluated by games coming into the market further down the road.
The true danger, in fact, is not in believing that single player and massively multiplayer are too similar but rather that they are not similar enough. The Sims Online is a virtual case-study of this sort of failure. Everyone expected it to bring in a flood of new players and revolutionize the genre. Why it didn't is what Paul Sage refers to as "the corruption factor":
They took something from these other games, an avatar that's you and you move around and everything else, and they took away what was fun out of the Sims to get to that point because they had this idea of what Massively Multiplayer Online games were and thought that it had to fit that rather than naturally extending The Sims which would have been a lot more fun I think for a lot of people so there's a trap there."
Chapter 4: The Stratification of Online Gaming
The "triple A" MMORPG titles are currently engaged in a feature war. Budgets are driving ever higher and higher as scopes soar to larger and larger heights. With that we are seeing a diversification of the genre. This move towards bigger and bigger games is leaving a window of opportunity. And within the window whole new types of games are being developed.
Long, Gaffney, and Sage explained the situation in detail:
And so I think that's already stratified itself into this layer of like you gotta spend three years and 20 million dollars or you're not going to be able to compete in that realm. But then I think there's going to be the smaller games like Tale in the Desert, Puzzle Pirates, and World War II Online where you spend hopefully not very much money because if you spend a lot of money on one of those products you're doomed. But hopefully you spend a very little amount of money and you attract a smaller, much smaller audience, you're going for an audience that's a few thousand versus a few hundred thousand. And you're going to make money off it and it's going to be like cable TV, hopefully, where there's generalized channels but you go get digital cable where there's not only the Discovery channel but there's also the Discovery Health channel so if you only want to see science shows about health topics you can and you know there's this ever increasing specialization for smaller and smaller niches of our populous so I think that's what we're going to see."
Gaffney: "And I think we'll see too those smaller games are going to have smaller budgets and I think there's a chance some of those will hit. I don't think a ton of those have hit in the last generation but I don't think that's endemic. I think we'll actually see more of those hit. And as an example take Counter-Strike. Counter-Strike has thousands and thousands of people logging in every day; they're playing the same damn twelve maps they've played since the game came out, especially now that Counter-Strike Source is out. And so there's a lot of the attributes you'd want out of an MMP right there but people are willing to accept very, very little content as part of that mechanism and that's our major cost. So I think PvP's going to be an example of that, because PvP games you can make cheaper than you can make player verses environment games, but it's only an example. It won't surprise me at all to see one of those small games take off in the next few years and make a lot of the big boys rethink where they're going."
Sage: "I think the first game that succeeds that is not a leveling based, kill the monster based game will bring in a completely new influx of players. And that's what we want to see. I think everybody in the room will get suddenly giddy by that."
But don't make the mistake, with all of this talk about smaller games, of thinking that the big games are going away. Peter Freese explains:
That's particularly true for the US development because the interesting phenomena we're seeing now is that a lot of these games are being developed in Asia and some of those are going to be westernized. And it's much cheaper to develop a huge amount of content in Asia than it is here because, literally, you can hire a small army of artists and designers for what we could pay a small team here. And that's going to be challenging."
Chapter 5: Innovation, evolving the genre
Time and again, we've come back to the term "innovation". It's a challenge that gaming faces over most other forms of entertainment. Few tout how their latest movie or novel innovates their respective industry. When a movie does come along and innovate it's huge; it gives us our Star Wars and our Matrix. It's a very defining moment to their entire industry.
Conversely games are expected to innovate. Even sequels are expected to bring something new to the table. Some of the most successful series of all time, the Ultimas, the Final Fantasys, the Grand Theft Autos, have gone back to the drawing board, sometimes radically, with every installment. Because games offer so many more entertainment hours than other products we need to provide a reason to play a new game. Few would wonder why you'd go see a new movie. You saw the last one. It's done. But games are many, many times longer and are often designed to be very repeatable experiences so they, inherently, are more challenging to get into people's hands. Why, after all, would a consumer pay for the same experience they already have? This is why innovation is one of the cornerstones of the industry.
MMORPGs, by extension, have an even greater challenge in this area. MMOGs are designed to support weeks, months, even years of gameplay. This greatly increases the investment of the player in a particular MMOG and makes innovation even more important as the enticements needed to convince a player to change games face an even higher bar than single player games.
While we've established that innovation is important there is another piece of the puzzle. It's easy to say, "Go forth, and innovate!" but it's also important to understand where innovation is available. Starr Long elaborates:
But is there anything a developer should be aware of when looking for these innovations? Long and Gaffney offered up a number of things which developers often overlook:
Gaffney: "But there's a trap there because it's kind of a clich? in the movie industry and, to an extent in our industry, when you're pitching a game you take two top sellers, you say 'It's going to be just like Myst plus Grand Theft Auto. Both of these games sold eight million units, we're going to sell sixteen!' So make sure you're innovating as well as stealing."
Long: "It's important that you're not just giving those platitudes. You have to say here's the specific feature of Grand Theft Auto: we're going to let you steal cars. Right. And here's the specific feature of Myst: once you get in the car you're going to have to solve an elaborate puzzle, that'll take twenty minutes to complete, to get it started."
Gaffney: "I don't think there are any magic bullets. Things that will scare me away pretty quick though is that a lot of people try to innovate by bringing up stuff that has been tried a hundred times, it's been tried in MUDs or MUSHes...
Long: "*cough* Perma-death *cough*"
Gaffney: "You stole my next sentence! People say "perma-death". We know what the implications are. There are really good arguments about why that has not succeeded in the market to date and people think that they're innovating by bringing up the stuff that hasn't worked. Now it's possible that there are takes on those things that are new and innovative but I never hear them. I always hear the same old stuff about 'We're going to have aging in the game and perma-death and blah, blah, blah.'
If it's not fun don't do it. It's not fun to spend five months on a character and then lose them due to a net bubble of lag that pops up. It's not fun to watch your cool hero turn into a decrepit old guy who can't even swing a sword anymore. And so why do I want to pay money to have this happen to me? I don't.
There's a whole bunch of, 'If it's been tried before and failed in the MUD-space we're probably going to point to fourteen ways,' 'Hey, Medivia tried that and it didn't work.' or whatever."
I can tell you forty-eight reasons why Lineage never would have succeeded, or would have been awful game design, and guess what: if I were the guy signing up that game then I would have turned down a hundred and fifty million dollar a year business.
So it's really easy to get cocky and be all "Oh we know all about this." But there's still room in this industry to surprise all of us. There has been in the last ten years and will be in the next ten."
Chapter 6: History, World Building, and Immersion
Creating a MMORPGs is one part game building, one part community building, and one part world building. That final aspect, world building, has gotten a lot of attention over the years and one of the biggest topics of discussion is how much the players should be allowed to affect the world.
Players naturally break things, sometimes maliciously, but often not. There have been raging arguments about what's appropriate in the fiction. This race or faction would never do this or that. Or players shouldn't be allowed to do something because of "The Lore".
The term Lore is a somewhat nebulous concept that is often attached to the entirety of the back story, fiction, and flavor based game design of an MMO. It's a concept that is all inclusive of anything which reflects an MMORPG as a virtual world rather than an online game.
The question, however, is whether this Lore is even important at all. Gaffney and Long had some very interesting things to say about how they view game histories when listening to pitches:
Long: "You know things that people usually think are really important, like the real elaborate fiction and storyline, are almost irrelevant to us compared to all the other things Jeremy has mentioned. Like game mechanics and technology, and yeah, it's nice if you have a good story, but for us that's much less important."
Gaffney: "In part because the player's going to write the interesting part of the story. Who cares about ancient history when you have history being made daily?
If there's too much of it, it's just going to be fiction getting in the way of people having fun and they're just going to want to push the escape key and get away from it. We really want to get to them playing and interacting in the world."
But players are more important. The goal, the target, the holy grail perhaps, is to create a world that the player is immersed in but where their stories are the focus. This means you have to strike a delicate balance, made more delicate still because you're catering to the likes of thousands or even hundreds of thousands of people. This balance must set a stage for the player while leaving them the main characters.
The danger is that if the player is not the focus the Lore gets in the way of the player playing the game and, by extension, having fun. However it is the environment that makes entering the world compelling. It provides consistency and is a mark of value. Without a well crafted world for the player to exist in your MMOG is just another game and you've missed out on one of the major selling points of the product. As in all things, it is the balance which is important. And as the genre grows and generations pass that balance will become more and more polished as it is a key aspect of the genre.
Chapter 7: The Spoken Word: Voice in Online Gaming
Technology is not particularly the focus of this article. However, one evolving technology is likely to become very important through the next generation and is already having a strong impact through third-party applications among the hardcore audiences. This technology: voice communication.
It is, perhaps, the one technology which has not really been addressed by any of the games that have come out in the First or Second Generations but which will have to be approached, one way or another in the Third.
Your average consumer, one, doesn't know how to type and, two, they don't want to. So I think what's going to end up happening if we want to appeal to broader audiences is we're going to have to use voice chat. I think you're going to have to have some kind of combination of text and voice chat, at least some sort of complex voice emote system. But it's going to have to be voice."
Gaffney: "An emote system might be a good system. Toontown did some good stuff with that. But my first reaction is: It's going to push our demographic up. Mom does not want her kid, sitting in the living room, chatting on a headset with some strange guy over the internet. It's just that simple.
With text you can at least record it better and filter it or whatever and you have none of that in voice. In fact it's very tough to even figure out how to mute people in voice. I play Counter-Strike all the time. I have to hit ninety-eight keys to figure out how to do it each time. I think these are the things that are going to have to improve, otherwise it's going to push our age demographic higher.
But it's such a key feature in really coordinating a combat and the most fun, I would argue, in MMPs to date is coordinating the combat. There's really no other game style you do that in except some of the shooters, which are the more niche shooters, and it's fun. Working as a team is fun. I love it when a plan comes together."
Long: "The reality is, whether we want to or not, or whether we think its technically feasible, people are doing it anyway. If the game doesn't support it they're doing it with third party tools. So why not include it because it's going to happen anyway?
Gaffney: "I think most games with voice chat don't do it as well as Teamspeak and so Teamspeak isn't going to be run out of the industry any time soon. In PC games Teamspeak has very little competition."
Weil: "Well DAoC and Ultima Online both said 'We're not even going to bother.' Everybody uses Teamspeak anyway so we're not going to bother.
Long: "Yeah, except, the problem is, again, if we're talking about expanding our market then your average consumer has no knowledge or ability to use a third party program. So I agree that for a power user, such as you and myself, Teamspeak is ideal. But for Joe Consumer it's the whole argument, 'Well we don't even need text chat in our games because there's IRC. So we should model it after IRC'. Again, IRC works great for us but for Joe Consumer..."
Weil "Ship them the earphones and tell 'em where to plug 'em in and there you go."
Long: "That's why Xbox Live is so successful."
But whatever it may be a developer should consider this an opportunity. Voice is one of the major open doors for innovation in the next generation and whoever tackles it well will do well in turn.
Chapter 8: Beta Testing, Millions Served
One of the unique features of MMO games, thanks to the Massively Multiplayer aspect, is the large scale beta test. In your average game, beta is a simple, internal, process of bug testing. But MMOGs are so large that you need to hit a certain critical mass of users to have any hope of finding even the majority of your bugs and balance issues.
The catch, of course, is that this critical mass is far too large to employ internally: somewhere on the order of thousands, even hundreds of thousands of testers when the time comes for load testing.
So how does one hope to manage such an endeavor? How do you get your game tested without giving your game away? Long broke it down into three major questions:
The pros and cons: Because if you do it too long you run the danger of a large segment of your potential consumer base consuming all the content. And you run the risk that when you start charging people go, 'Well, I'm done, you gave it all to me free, so why should I pay you now?' But that's balanced with the fact that you absolutely have to get them in there to test the stuff because there's a critical mass in these kinds of games that are required to work out all the kinks, to find all the problems.
But then there's a whole other layer to it about how do you incentivize people to tell you about those problems? There's an incentive for them to tell you about bugs that involve stability, but there's not necessarily an incentive for your beta testers to tell you about bugs that are exploits or are balancing issues if they can exploit them to their advantage. And so in almost every single game there's a group of people who find exploit bugs and keep them to themselves until we find out on our own or someone rats them out.
I don't think as an industry we have a really good answer yet on any of these questions: 'How long?', 'How many people?', and 'How do you incentivize people to tell you about the problems?' And so I will gladly pay ten million dollars to the first person who has all these answers nailed."
Weil: "And I don't think you'll get any consensus that there's a right way to do it."
Sage: "That's right, because I think the beta test is directly related to the game and there's no good thing about these are the rules for a beta test. It's all about the game itself. And certain games will have things that work for them."
The first decision is positive or negative reinforcement (or both). Will you give your testers some sort of bonus if they report a bug? Typically this would be an in game bonus. The advantage here is that, suddenly, it's in a player's best interest to report a bug. They get something out of it that is, to them, tangible. Of course there are flaws. What if, for example, the bug allows them to duplicate in-game currency? Suddenly they have an infinite source of cash. Now say your reward is in-game currency. Why would the player report the bug? They can report the bug and gain a finite reward or abuse it and receive an infinite reward.
Some of the most successful rewards seem to be things that are exclusively created for reward purposes. Things like giving the player's character's titles or special items, or things of that nature. The danger is that either other people will scream about favoritism or they will become desperate to get the rewards themselves and will bombard you with hundreds if not thousands of irrelevant bug reports, making the whole process much more difficult from a development end.
Negative reinforcement hopes to intimidate the player-base into reporting bugs. The typical response is to ban, or at least suspend, accounts where the developer finds a player that has been aware of a bug and willfully failed to report it (and, typically, abused it themselves). The hope is that players will value their ability to play the game and thus will report bugs they find so that they don't risk losing that privilege.
The problem here, however, is that there will always be those who will take the risk and abuse bugs anyway. Still more will just quietly save bugs and hope that they're still there when you release. If you are too easy on these people then more will follow in their footsteps. They'll rationalize, "Oh, well those people abused a huge exploit and only got a slap on the wrist. They'd never ban me." Conversely, if you are too aggressive then your testing community may begin to feel oppressed and either begin to generate a negative buzz about your game or even, in a worst case scenario, begin to do things that are directly harmful to the testing process.
It is possible that the best method lies somewhere in-between. However, as Paul Sage pointed out, every beta test is different. There's no clearly right answer as it depends entirely on the mechanics of the game, the goals of testing, and the community in question.
As for the other questions, Gaffney, Freese, and Weil addressed Long's question of "How long" in a little more detail:
Gaffney: "I think it's interesting that WoW's had such a long test period and we'll see. Regardless of whether people who buy WoW will play it for three months or six or twelve it'll be shorter if they got a bunch of those months for free unless it's just going to keep them forever.
The problem with the short open beta is you just need to have, as a company, the balls that if you do find big issues in that short open beta to shut the beta down and go fix them and then have another beta instead of extending beta while you try to fix them."
Weil: "I don't know if my perception matches reality at all but I thought that the way City of Heroes did their late stages of beta was a good one. You know, we're going to turn on the game for forty-eight hours. Everyone slams it and plays for forty-eight hours. Ok, we're turning it off. And they turn it off. And so the emphasis is getting the data, analyzing the data, and fixing the problems rather than 'we gotta keep it up, we gotta keep it up' so we have to keep the servers up the whole time. A lot of man power goes into that."
The point, then, is not that there is one right way but just the opposite. A company has a lot to gain by finding the perfect solution for the game they're making, be it tried and true or totally revolutionary. There's a lot of wiggle room and success breeds not only a better game but also is one of the major pieces of marketing and buzz that will accompany your game into launch and beyond. Sometimes even significantly beyond.
There is a further consideration of what types of players you allow into your test. Beyond the obviously negative elements who simply exist to cause grief either for your other players or for you as a developer there are two types of players who can actually harm the development process through their divergent interests.
The first is the "fanboy" mentality. This is the type of the player who is, for one reason or another, absolutely sold on the game and who is absolutely convinced that this game will be the best game since sliced bread. While they can be good for hype, they can also have a serious negative impact upon the game's testing community. Quite often they feel the need to defend the game, and, by extension, their love for it, and will actively debate or, more likely, attack those testers who are making comments about negative aspects of the game. While some of those comments may be off base the important point is that they can prevent useful information from ever reaching the developer's ears. It's pretty easy for them to create such a cloudy signal to noise ratio that actual useful messages are lost in the clutter.
Gaffney mentioned one of the more interesting types of "fanboys":
There's a crowd of gypsies right now moving from game to game to game who have all these hardcore opinions formed from their early things. Bartle did a really interesting article on this, and because they're in the betas and they're vocal they're setting the trends for a lot of these games and they're limiting innovation too."
Well, like everything else, that depends. There are dangers in allowing these types of players into your game. You're getting skewed reporting and input that's going to be irrelevant or even counter to the goals of 95+% of your player base. On the other hand the information you can garnish from how these highly experienced and organized groups can attack your game and your content can be invaluable.
The moral is simply to understand who you're letting in the door and monitor them appropriately. The biggest danger is that most developers are hardcore players themselves. It's easy to hear the praise of this ultra-hardcore audience and think you're on the right path because you like it too. It's what you want to hear. But this is the road to hubris because Joe Consumer is not hardcore. He has an entirely different set of needs than the hardcore minority and, ideally, you need to balance your game design to fit both as much as possible.
Chapter 9: Hype and the Release of Information
One of the interesting questions about games is marketing. This is especially true in MMO games for two reasons. One: your beta test. This relatively public and often long test period will generate a massive amount of hype, good or bad, depending on how it goes. The second is the very nature of your audience. By definition every player has internet access. They know how to browse the web, and they know how to look for news. While they may not be advanced enough or interested enough to worry about third party programs and other stumbling blocks they most definitely are better informed than the fan base of any other genre.
What this means is that the flow of information must begin earlier and be more carefully monitored than in other genres. World of Warcraft and Everquest II are very interesting studies here as they have developed virtually in parallel. Gaffney addresses this:
"I think if your game goes to beta at poor quality you probably want to keep your vest tight. I think people might figure out that, hey, games that keep their vest tight for a long time are of poorer quality and they'll start assuming that. I don't think you should go to beta when you have that kind of issue. I think that's why you stay in closed beta unless you're forced to by some sort of fiscal year or some other silly thing like that."
Strasz: "I think that if we use single player games as an example than Fable's been a really good example of talking about design early on and then later having to deliver different stuff. But kind of thinking of things in terms of single player, and this is kind of unfortunate, but you don't have these kind of open giant beta tests for single player games so for something like Halo 2 there's a lot of secrecy and Half-Life 2 everyone is anticipating it. The buzz is huge. They want to see what the game's going to be like but you don't get that luxury with large open beta tests, so that's also an interesting aspect to the NDA. It's not review day just when it gets into reviewer's hands."
Fable is an excellent example, online or not. Fable was a good game that received harsh criticism, unnecessarily in some ways, because it had build some of the biggest hype in gaming history thanks to all the things that were announced very early in development. These unfulfilled promises hurt an otherwise quality title and cost Fable more than they gained to the point where Peter Molyneux issued a public apology for the hype verses the reality of Fable.
It's hard. Most developers are necessarily fanatical. When you're really excited about something it's natural to want to talk about it. Despite that there is a general guideline here if not an actual rule: Talk about things that are in your game. If you do that simple thing you avoid over inflated hype yet still generate buzz. As long as you don't shoot yourself in the foot, as Gaffney states, game quality will win out.
Chapter 10: Community, Putting the Massive in the Multiplayer
Community is something that a good gam