the game life cycle
Key Chapter Questions
- What is the definition of a bug?
- What are the primary severity levels associated with bugs?
- What are the phases in the standard life cycle of a game?
- What is the distinction between closed and open beta?
- What are the distinctions between different testing disciplines?
Game testing might seem simple and self-contained, but the truth is that testing is akin to engineering and programming. No wonder the official title for game testers is sometimes “Jr. Engineer”! Some of you will actually need programming skills to succeed at testing. Others will need an eye for detail—or the succinct prose of a journalist. In this chapter, we’ll uncover the world of bugs. We’ll also tackle definitions and severity levels, and examine how testing relates to the life cycle of a game—while looking at various testing methodologies.
Bugs Defined
Let’s hear it for the dictionary definition! According to Merriam-Webster, a bug can be many things:
- a) an insect or other creeping or crawling invertebrate (as a spider or centipede)
b) any of several insects (as the bedbug or cockroach) commonly considered obnoxious
c) any of an order (Hemiptera and especially its suborder Heteroptera) of insects that have sucking mouthparts, forewings thickened at the base, and incomplete metamorphosis and are often economic pests—called also true bug
- an unexpected defect, fault, flaw, or imperfection (the software was full of bugs)
- a) a germ or microorganism especially when causing disease
b) an unspecified or nonspecific sickness usually presumed due to a bug
- a sudden enthusiasm
- enthusiast (a camera bug)
- a prominent person
- a crazy person
- a concealed listening device
- a weight allowance given to apprentice jockeys
Definition #2 is the winner here; since a video game is a form of software, it fits like a glove. Let’s take a closer look at this definition:
- unexpected: No one creates bugs on purpose; if it seems like it was done on purpose, it’s an Easter egg—a different animal altogether.
- defect, fault, flaw, or imperfection: Something is wrong; it makes the game worse, ugly, or unplayable.
In games, every element that does not enhance the game but detracts from it is a bug. Sometimes a bug is almost im-possible to see—and sometimes it’s right in your face, taking up the entire screen. Or maybe it’s an audio bug and you’ll hear it if the 5.1 surround sound is at full volume, shaking the whole building. When you’re a game tester, it’s your job to locate all the bugs you can, replicate them—make them occur again, at will—and write a report for the developers. You are the eyes and ears for the game they are making.
Your main purpose as a tester is to:
- find bugs
- replicate them
- report them
Secondary objectives are to:
- verify that the bugs have been fixed
- make sure the game is fun
If you keep these directives in mind, you’ll become an extraordinary tester. Now it’s time to take the first step into the “inner circle” of testing. Let’s start by taking a closer look at bug severity levels.
Edward Rotberg on the Tester Who Saved the Day
Ed Rotberg has been in the game business since 1978, working as a producer, designer, programmer, and executive manager at various times in his career. He was one of the founders of Videa, which was eventually acquired by Bally Midway. His career has also included a two-year stint at Apple Computer. He has worked at Atari (Atari Baseball, BattleZone, Star Wars, Blasteroids, S.T.U.N. Runner, Shuuz, Steel Talons, Guardians of the Hood), Bally Midway (Snake Pit), 3DO (Station Invasion, IMSA Racing [M2]), Silicon Entertainment (NASCAR Silicon Motor Speedway), THQ (MX Superfly, WWE Crush-Hour), and Mine Shaft Entertainment (High Heat Baseball 2002, Nokia Pro Series Golf, Blazing Angels: Squadrons of WWII).
While I was at 3DO, we were doing an “edutainment” title on a rush schedule. There was a lot of video content to the game, which was targeted toward grade school children. In one of the video sections that was parodying a TV talk show, the audience (not the player) was urged to call an 800 phone number that spelled out a cute and appropriate phrase. Fortunately, on his own initiative, one of our testers decided to call this number. It turned out to be an adult hot line, and we had just enough time to change that video content before the game shipped.
Bug Severity Levels
To get things going, let’s first learn about the four different severity levels associated with bugs. The nomenclature may differ from developer to developer, but respective meanings are virtually identical.
Low Priority Bugs
Low priority bugs (or low bugs) hardly matter; in fact, it often makes no difference to the development team whether they are fixed or not. Examples of low bugs are minor graphical glitches, a short sound distortion here and there, some small error in the user interface (including a typo, which would personally frustrate us!)—any “error” that does not impact the flow or the way the game is perceived in a major way. Low bugs abound in every single game you’ve played, from low-budget to AAA; they are sometimes ignored in favor of more serious bugs—the ones that do make a difference.
Medium Priority Bugs
Medium priority bugs (or medium bugs) should be fixed. They occur more often than low bugs—and they can almost always be counted on to happen. So if a visual bug happens often, instead of rarely, it might be upgraded to medium. At the same time, a bug can be categorized as low if it occurs deep in the user interface; the same bug might be a medium bug if it occurs on the title screen. Another example is a bug that would annoy a player, but not affect gameplay.
High Priority Bugs
High priority bugs (or high bugs) must be fixed. A game that ships with high bugs is either a bad game or it has been rushed to market. High bugs seriously affect gameplay; they make everything stop in its tracks. If you can’t open a door and that same door ends the level, you got yourself a high bug. If you can’t change weapons, that’s a high bug. If the frame rate drops to 20 fps instead of the usual 60, guess what? It’s a high bug.
Bugs That Won’t Die
The longevity of some bugs amazes me. I’ve had players of MUD2 complain about some strange but obscure goings on which, when I finally tracked them down, were in major pieces of code that have been executed tens of thousands of times a second for 15 years. It’s astounding that the program ever worked at all, let alone that it managed to do so robustly for so long. This isn’t an isolated incident, either; it’s happened at least half a dozen times. I look at the code and think, “Why didn’t it instantly crash?!”—but this somehow managed to be avoided.
Dr. Richard Allen Bartle (Visiting Professor in Computer Game Design, University of Essex)
Critical Bugs
Critical bugs are very special. A critical bug demands immediate attention from the developers. Nobody goes home—and if they’re already home, they better come running back. Critical bugs are responsible for “the sky is falling” 2:00am phone calls and other general chaos! But what is a critical bug, really? Critical bugs cause “disasters” such as crashes, freezes, and data corruption. If you ever had a 50-hour saved game corrupted in one sweep, you know what we mean. Critical bugs must be fixed at all costs—and if the publisher spots a critical bug during compliance testing (discussed later in this chapter), the game is sent back to the developer.
Testing at Every Stage
Testing is a critical component to the process. In fact, you should test at every stage of development. At NCsoft, QA worked directly inside our Scrum process teams (which meet frequently and quickly to address problems and update everyone on progress) and tested content as it was being created before we even sent the game on to publishing QA.
Starr Long (Executive Producer, The Walt Disney Company)
Page 2
|
|
Southwest Gaming Expo
