Jump to content

  • Log In with Google      Sign In   
  • Create Account


Like
1Likes
Dislike

Bigger Than Big: The Game Audio Explosion

By Steve Kutay | Published May 01 2006 06:02 AM in Music and Sound

If you find this article contains errors or problems rendering it unreadable (missing images or files, mangled code, improper text formatting, etc) please contact the editor so corrections can be made. Thank you for helping us improve this resource



I. Far Beyond Bleeps and Loops

The new console era is upon us. It has been met by developers everywhere with great anticipation, promise, …and yet, reluctance. Programmers have spent a large portion of the past decade
squeezing every last bit of potential from our PS2s, Xboxes and Gamecubes. Now, after tricking these machines into performing beyond their expectations, the shackles of technology have been lifted
yet again. But will the next generation consoles guarantee better audio?


No. We can certainly expect more audio due to an increase in available memory, and the ability to add additional content within BD-ROM and dual layer DVD-ROM formats. But what makes audio sound
good doesn't necessarily have anything to do with performance and delivery specs. Surely, our ability to manipulate audio will improve, but it will mean nothing if the content doesn't deliver. This
article focuses on sound creation, and will enable you to pave the way for effective and successful interactive game sound.


You have the ability to put the creative spark in motion and keep it moving regardless of which game format you are developing. Knowing and preparing your sound team as well as understanding the
processes through which they work, will ultimately help you to keep the audio on track, both artistically and financially.


II. The Audio Team

A few years back, I was scoring a short animated film. One of the animators for this film held a day job at a well-known entertainment company that had just released a CG movie about dinosaurs. I
asked him what he did on that project, to which he replied, "I did all the toenails."


I couldn't help but think of the army of people responsible for the teeth, eyes, scales, and so on. None-the-less, I saw the movie and it was visually stunning. Realistically, game budgets will
not allow for such an extravagant audio team, but it does illustrate a good principle: that your audio personnel have well-defined roles with which to focus their efforts. Collectively, your audio
will be that much better for it.


Game budgets once mandated that production costs stay low, so it wasn't unusual to find that one or two people produced all of a game's audio. Today, the stakes are much higher, and so are the
budgets. Consumer expectations have grown, requiring a movie-like experience within the confines of their homes. The interactive market has become a battlefield for franchise superiority. Bland,
over-used audio must not be the exposed link in the armor of any publisher or developer.


Whether you are using an in-house audio department or outsourcing the audio completely, it is important that individuals have well-defined roles that do not cross over into the other aspects of
sound production. If the Audio Director is splitting time as the Sound Designer, and the Sound Designer is also the Composer, you can be sure that none of these shared jobs will get the proper
attention they require. It is important to obtain a list of your entire audio team that breaks down the responsibilities of each member. Use your sound budget to fortify any areas in sound production
that need particular emphasis. We will discuss more on budgets later, but for now let's start at the beginning.


III. Communicating the Vision: Pre-production

A. Early Bird Catches the Worm

By their very nature, creative people are passionate about what they do. You shouldn't have difficulty finding the enthusiasm amongst your sound team. Yet this inherent motivation is not something
to be left without guidance. You will be doing your budget as well as your team's morale, a disservice by letting your sound team simply "have at it". When it comes time to add sound, the sound
designers have both an advantage and a disadvantage compared to the other production team members.


The advantage is, that by the time the game is ready for audio creation, the game has taken real shape and personality. This helps to guide the direction of the sound effects design. The
disadvantage is, that since the sound design is one of the last stages to be developed, previously fallen deadlines become the responsibility of the sound design team to make up. By bringing your
sound designers up to speed early, you can avoid costly third and fourth revisions.


B. The Documents, Please!

Giving the sound team the most recent build to play, only gives them a partial picture of the artistic direction of the game. The sound team, like the art department, must understand the
metamorphosis of the game's characters and landscapes.


To do this, compile a book or digital archive that chronologically depicts the artwork, from the earliest sketches to the final in-game representations. Arrange an in-depth meeting between the
sound designers, composer and the Art Director to discuss the game's development from an artistic standpoint. This will help your audio team create the proper palette of sounds in much the same way
an artist creates a palette of colors.


For story-driven games, distributing copies of the script will be necessary to illustrate the motivation and goal of the game. While this is critical for composers, the sound designers will
benefit by the added sense of immersion into the game.


Perhaps the best form of communicating the vision will come from the Game Designer. The game designer works tirelessly in his pursuit to create "the best game ever". He is never short of words
when describing the intent of the game. Though his work is creative, his methods are mostly technical. No one understands the abilities of the characters in such detail as the game designer, as the
great number of technical documents he produces will attest. These documents are invaluable to the audio team. By thoroughly examining level overviews and enemy specs, both sound designers and
composers can create complimentary aural depictions. Bosses that are slow but powerful, or enemies that are stealthy will be revealed in great detail within these documents, providing the backdrop
from which the sound designers can create.


C. The Demos - Getting on the Same Page

Once the above preproduction steps have been completed, it's time for the sound design team and composer to begin creating demos from game capture. Create three to four movies 60 to 90 seconds in
length from different levels in the game. Be sure to include the ambient portion prior to the action in order to hear the game shift from low to high levels of activity. However, this may not be
possible for some arcade style games.


Once the sound design and music are complete, a mix of all the audio content should be performed by the Sound Lead or Audio Director in either stereo, surround or both, and exported with the
movies for review.


It is important to have in place a team of reviewers that appropriately represent those who have creative input. These might include, but are not limited to, the Developing Producer, Publishing
Producer, Executive Producer, Associate Producer, Game Designer, Art Director, Audio Director and a franchise representative if applicable. A robust review team will help generate an accurate and
collective review. If changes in the demonstration audio are required and then subsequently agreed upon, your audio is ready for production.


IV. Sound Design Production

A. Emotional Response to Sound

From the beginning we have been programmed to respond to sound. A mother's voice, a church bell, or police sirens conjure an emotional response. Sounds help us to decipher the world around us.
They warn us of danger, call us to action and bring peace and tranquility to our lives. The more expressive the sound is, the greater our emotional response to it. Sound effects correctly placed in a
game should evoke this response while defining the environment, circumstance and personas on screen. Due to the random nature by which sounds are triggered in a game, they must effectively co-exist
without losing definition or character when multiple sounds occur in close proximity to each other. Let us examine some general observations in game sound design.


B. Beware of Sonic Sludge!

There is a finite amount of sound data that the ear can properly interpret before fatigue sets in. It is the role of the sound programmer or director to prioritize which sounds are most important
and at what times they are important. The sound designer on the other hand, must always create content that will be effective, regardless of the circumstances that exist at the time a sound is
played. Good sound effects should work well alone and in combination with many other sounds. This is a challenging task, but careful forethought and planning will produce a rich, dynamic and
satisfying interactive soundscape.


The key to preventing sonic fatigue is to create sound effects that vary in volume and frequency in relation to each other. A single sound effect that is loud and contains equal amounts of low,
middle and high frequencies may be effective when played alone, but if all the sound effects are loud and contain a similar frequency spectrum, it becomes difficult to decipher one sound from the
next.


In most cases, the sound designer delivers the sounds at a reasonably loud volume, to allow the audio director or programmer to appropriately mix those sounds into the game, setting the playback
volume for each sound. However, it is the job of the sound designer to emphasize different frequencies according to the requirements of each sound. To do this, the designer must know which sounds are
likely to be played together at any given time, then selectively decide which sounds will emphasize specific frequencies. Higher frequencies provide detail. Upper middle frequencies provide presence,
while lower frequencies depict power or energy. Too much emphasis on high and upper-middle frequencies will lead to fatigue, while too many sounds containing lower or sub frequencies, will become
muddy and detract from the overall detail of the sound design. The goal is to create individual sounds that do not compete, but compliment. With this in mind, the sound designer must appropriately
focus on the frequencies that will best suit each sound effect. This process essentially carves out any unnecessary sound space to allow additional room for other sound effects to be heard. When
volumes and frequencies are selectively assigned, the sound effects will breathe and compliment each other regardless of when they play.


C. Pacing - Building Toward Climactic Moments

Now let's examine the sound design from the "Big Picture" perspective. Game and level design documents will provide the structure of the game in terms of moments of emphasis. Generally, these
structures take the form of peaks and valleys that convey changes in difficulty as the game progresses. Usually, the peaks represent a boss fight, though not necessarily so. When examined as a whole,
the sound design should appropriately compliment these arching structures, and allow, from a sound perspective, a sense of building toward these peak moments. If the sound designer has examined the
enemies and situations thoroughly, the overall sound design will naturally fall into place, appropriately following the peaks and valleys within the game. However, if for example, minions sound as
powerful as bosses, some adjustment will be necessary to bring down the emphasis of these weaker and less difficult enemies. By not doing so will result in sound design that does not match the
arching pattern of the game. To put it simply, there can be "too much of a good thing". Let's now look at the specific areas of game sound design.


D. Ambience - Defining Environments Through Sound

Initially, ambient sound should effectively portray the setting, location and time frame of the game or its various levels. For instance, percussion and double reed music, a multitude of bartering
voices and distant clanking iron would suggest a medieval marketplace. As the game progresses the role of the ambient sound is to support the circumstances with which the player is involved. Does the
sound within the environment evoke danger or safety? Activity or inactivity? Conversely, ambience can be used to deceive the player through suggesting a false circumstance, such as creating a sense
of calm before an ambush. Under all these conditions, good ambient sound should portray a living environment.


The psychological impact of ambient sounds can add much to the onscreen imagery, though not physically present in the scenery. For instance a distant, sustained cry of an infant suggests
vulnerability or insecurity. A broken fence rattling in the wind of an abandoned city, suggests to the player a previous traumatic event. These are subtle examples used to arouse awareness in the
player. More obvious sounds should be used to cue the player of his direct proximity to danger. Dark drones or muffled enemy vocalizations will prepare the player for fierce combat ahead. Fear,
anticipation and anxiety are easily evoked by the careful placement of ambient sounds.


E. Impacts and Destruction - Breathing Death into the Non-living

Early on, comic books depicted the sound of the action scenes through the use of words that sonically mimicked the action. Over time, words like "thud" "pow" and "zap" lost their effectiveness.
Comic book writers had to jog their imaginations to express sounds in more creative and exciting ways, such as "Kathwaaap', "fwuuuhmp" and so on. Similarly, the sound effects in early games
experienced a renaissance as memory increased and streaming technology allowed for more and varied sounds to be launched under the animations. However, no increase in playback performance will ensure
the effectiveness of the sound effects, if the sounds are not expressive.


From a sound perspective, impacts and destruction must primarily convey suffering and submission. These terms apply naturally to the vocal efforts triggered under an opponent or avatar under
attack, but are more abstract when applied to inanimate objects. Since the human voice is the most expressive instrument in existence, applying human-like characteristics to the 'non-living', will
help give the sounds a more life-like and expressive quality. Twisting, screeching metal, the deep thud and release of broken concrete and wood that creaks, pops and splinters convey expressive
responses to the forces applied to them, in much the same way a grunt, moan and exhale expresses human injury.


Additionally, impacts and destruction sounds should proportionately depict the transference of energy between the weapon and the target. A metallic ping with a ricochet is an effective response to
a bullet on metal, in which the transfer of energy between a low-mass object at high speed can be observed. A missile explosion, on the other hand, is more powerful and slower to develop, therefore
requiring an equally proportionate response. The sound of larger impacts with destruction should develop through three basic phases: Attack, Sustain and Release.


The Attack is the first and shortest event of the three. It is important to note that the attack is not the sound of the weapon or projectile. In this case, a missile,
contains it's own dry explosion sound that is launched under the animation of the missile explosion. Therefore the attack will be the impact sound based on the material composition of the target.
Since the attack and the dry explosion of the missile will happen simultaneously, the attack should have a short period of 'lead-in' or silence to allow the peak, or initial part of the
explosion of the missile to be heard uncompromised by the attack of the material impact.


Next is the Sustain, which introduces the debris and material breakdown created by the explosion. Over this phase, detail should be observed. The sustain should sound less dense
than the attack so that the specific details of the destruction can adequately be heard.


The final phase is the Release, which is a response to the destruction that should characterize a kind of 'submission'. This phase of the destruction should contain lighter
falling debris based on the materials destroyed, movement of dust and earth and perhaps steam.


When all three of these phases are exhibited, the destructions will sound more expressive and compliment the weapons by adequately portraying their explosive energy.


For "The Incredible Hulk - Ultimate Destruction" we maximized the detail and movement of large, explosive forces by dynamically altering the stereo field throughout the three phases of
the destruction. The attack phase was almost entirely monophonic, while a quickly widening stereo field was applied to the sustain, finally resting on a wide and fixed stereo field for the release.
The result was destruction that moved rapidly over a wide area, thereby adequately portraying the Hulk's enormous power.


F. Weapons - Know Thyself, Know Thy Enemy!

It is a lesser-known fact that a gunshot at close range, sounds less threatening than from 40 or even 80 yards away. Since most people have never fired a gun, their expectations for the sound of
gunshots as depicted by the entertainment media are very high. Therefore, even in games based on historical simulation, some amount of sonic sweetening will be necessary. In the case of a
"period" war game, multiple recordings of the specific weapon should be blended together to create a satisfying gunshot. These might include mixing together the various distances recorded for the
gunshot, as well as the dry trigger and shell discharge sounds for the specific firearm. Sounds created this way will be sonically interesting while retaining the historical accuracy of the
weapon.


For science-fiction or fantasy games, the imagination is the sound designer's only limitation. As mentioned previously, the design documents will shed light on the abilities of the enemies and
characters within the game. The weapons detailed in this document should explain the amount of damage incurred by each weapon. It is important that these sounds appropriately match the damage
potential, since the player will, to some extent, be judging the amount of damage from each weapon by the sound it creates. For example, weapons that contain a charge-up sound before firing,
indicates to the player that a great amount of force is forthcoming. Likewise a weapon that produces a large discharge noise would produce the same result.


From a stylistic perspective, weapons are an extension of the personalities of each character and should compliment the character's physical attributes, abilities and in some cases, their heritage
or history. For instance, the sounds of swords, knives and shuriken should be as stealthy as the master ninja who wields them. The character of these sounds should compliment the physical qualities
exhibited by the ninja and reflect the mastery of the ninja tradition. With this in mind you should expect the sounds to be light but fierce, focused and evoke quickness of movement.


G. Vehicles

Since vehicle sounds typically respond to controller movements, and not animations, they can be difficult to perform in a plausible manner. Developers for racing games are likely to have
robust code for manipulating vehicle sounds. Since we are focusing on sound production, and not programming, let's examine the basic elements that make up vehicle sounds.


In most cases the sound designer will provide four separate engine sounds per vehicle: an idle loop, acceleration, a steady thrust loop and a deceleration (engine decompression or braking). The
idle will simply indicate that the vehicle is engaged. The acceleration and deceleration sounds should be designed to seamlessly crossfade into, and out of the steady thrust loop via programming.
This formula is effective for simple vehicles with a low threshold of speed in which the vehicle will quickly reach maximum velocity until the button or trigger is released.


If the visual perspectives of the vehicle can be changed, so too should the sounds that accompany the vehicle. This will ensure a greater sense of realism. For instance, if inside and outside
perspectives are available, subtle shifts in the observed engine sounds should be present to support the change in perspective. An inside perspective will result in a de-emphasis of the higher
frequencies that are present within the engine sounds, giving those sounds the muffled quality one would expect when listening to the engine from inside. One way to perform this, is for the sound
designer to supply separate versions of the engine sounds based on the perspective observed. If the sound designer has access to recordings from the various perspectives, this will be easy to supply.
However if these sound perspectives are not available, or if the vehicle is fictitious, separate mixes that include changes in equalization should be performed in order to support the visual
perspectives.


For added realism, intermittent sounds can be supplied to add feedback based on the driving conditions or the state of the vehicle while operating. For instance, wheel-based vehicles will contain
surface noises used to indicate the terrain (tarmac, gravel etc.). Metallic rattling and scraping is used to indicate the state of a vehicle that is damaged. The addition of these and other
intermittent sounds add a heightened sense of realism and immersion when operating the vehicle.


H. Menus - Less is More

As games have become more sophisticated, so too have the menus. Players can customize a variety of options as well as view or purchase an array of unlock-able content. This, of course requires
more navigation. In most cases, sounds will accompany the navigation to provide greater sensory feedback. No matter how enjoyable these sounds may be, their repetition will soon become an annoyance.
It is always safe to create short and subtle sonic events to accompany the menu navigation, so that the sound is felt rather than heard. This way the player can benefit from the additional sensory
feedback, without being discouraged by the mere repetition of the sounds.


V. Music Production

A. The Underscore - Interactive vs. Film Music

Until recently, you simply could not compare game music to film music. Every aspect involved in their production, from budgets to performance, made it an impractical comparison. Today, these two
media have a working relationship. Games are created to support movies and movies are made from successful game franchises. Film composers are now writing for games, and some game composers have made
the transition to film. Hollywood orchestras and orchestrators are now commonly used for game music scores. Why make this comparison? Because even though the considerations involved in their creation
are different, their effect and function are relatively the same.


B. Story Driven/Roleplaying Games

As the name suggests, the scores to story-driven games must primarily tell a story. To tell a story musically is a sublime art. A composer must be well versed in the work of his predecessors in
order to understand what constitutes successful story telling using the language of music. Fortunately, centuries of music have been written for this purpose, allowing today's composer a foundation
for developing this art. We now associate certain sonorities and rhythms with specific actions, emotions or locations. Compositions like Rossini's 'William Tell', Wagner's 'Tristan' and Holst's
'Planets' have laid the groundwork for these non-verbal associations. Film and television composers have since expanded on these motifs to help express the elements within a story.


A portion of story telling is to define the environment, both time and place. Musically, we draw influence from folk traditions for such a purpose. Through ethnomusicology we can effectively
represent locations and time periods by incorporating traditional instruments, modes and progressions into the score. For instance, a tabla, tambour or sitar is appropriate for describing an Indian
location. If such instruments are not available, the music may be orchestrated in such a way as to mimic these traditional sounds. A modern orchestra is greatly enhanced by the addition of folk
elements for the purpose of describing a specific time and place.


Characters within a story are supported through the development of melodic themes and motifs associated with each character. Orchestrating the motifs throughout various instruments will provide a
sense of character development as the game progresses. In addition, varying the harmonic support of these themes will reflect the character's physical, mental and emotional states.


Game music for the story and role genre must highlight the dramatic events in the story as well as drive the game-play. NIS and FMVs are the primary tools for advancing the storyline and scoring
to these videos is generally a straightforward process. You must consider, however, that game-play is also a dramatic event that contributes to the overall development of the story. Herein lies the
careful balancing act of supporting the story as well as the action, without the music sounding repetitious. Cross fading alternate versions and transitions, or layering individual tracks that are
programmatically muted and un-muted, will secure the musical effectiveness over long periods. The programming methods of manipulating music within a game are beyond the intent of this article.
Further reading from game development resources such as GameDev.net will provide a closer look at some of the programming methods used in game music playback.


B. Action/Arcade and Sports Games

The most basic function of game music is best exhibited in 'arcade' style games in which the overall gaming experience is enhanced by the addition of adrenaline-surging music. The music helps to
drive the action, thereby heightening the intensity of the experience. For this reason, it's very common for these games to license tracks from well-known, marketable artists with a track record of
producing music that translates to the listener. The interactive potential of this music, has thus far been very low. However, as many artists are also avid gamers, they are beginning to show
interest in lending their talent toward interactive soundtrack design, if not producing tracks in their entirety.


Generally speaking, the interactivity of the music in arcade-style games rarely moves beyond loops and stings. In many cases, this is all that is required. However, as the complexity of
arcade-style games grow, so must the level of musical interactivity. The music for these games should support any changes in game-play. Power-ups, signature moves and multiple damage are all examples
commonly reserved for the sound design to immerse the player in the action, but are appropriately expressed through music as well. A deep understanding of the game-play will reveal to the composer,
new areas to interactively enhance an otherwise monotonous arcade soundtrack.


VI. Full Motion Video (FMV)

Since the FMV is a controlled environment, it is tempting for the sound designer to elaborate on the sound effects. While in some cases, it may be appropriate to heighten the dramatic impact of
the story; great care should be maintained to be consistent with the in-game sound design. An incredible-sounding FMV is surely a joy to behold, however, if the in-game sounds do not hold up to the
FMVs, the playing experience will be diminished. The purpose of the FMV is to dramatically move the storyline, and to provide a break in the action. Since Most FMVs occur after completing a level,
there is an inherent sense of reward when viewing the FMV. The sound design should pay respect to this as long as it doesn't stray too far from the in-game sound. The FMV should act as a seamless
transition into and out of the game play. In my opinion, it is best to use in-game sounds within the FMV wherever in-game movements or actions are present.


The second consideration for FMV sound is the mix of all the sound elements. All dialog, sound effects and music should be mixed at comparable levels to the in-game mix, unless there is a dramatic
motivation for stressing one over the other.


VII. Sound Reviews

The Beta date is just around the corner. Your sound team has worked countless hours, and is nearing the finish line. You might think it's time to examine the sound for any necessary revisions.
Well by this time it's probably too late. As mentioned earlier, the sound team is generally the last in line to begin creating their content. Add to this, the fact that all previously missed
deadlines becomes their burden to make-up. Your sound team will likely be delivering content right up to the last minute. You will need to have in place a regular and effective reviewing mechanism to
stay on top of the direction of the game sound.


Using the same group of reviewers used for the demo phase, create a questionnaire that rates the general aspects of the sound. Rating each individual sound would be time-consuming, so use
categories of sounds and include room for comments or explanations. By assembling the various questionnaires, you will be able to develop a consensus opinion that will reveal spots that need further
attention. If this is performed in a timely and periodic fashion, your sound team will be best able to manage the revisions, as they are needed.


VIII. Got Your Sound Budget? Use it!

A. Your Simple Checklist

Today's games are competing with each other on every level. Sound is no exception. You must secure the best resources possible for your sound team. This will require that you use your budget
wisely, and use all of it.


Prior to beginning the sound effects production, ask yourself the following questions.


  1. Is your sound team complete? (i.e. sound designer(s), supervising/Lead sound designer, composer, audio director and audio programmer)
  2. Is each member of the sound team assigned a specific task uncompromised by additional or overlapping roles?
  3. Is your sound team assigned only to your project?
  4. Does your sound team have enough time to complete your project?
  5. Does your sound team have the adequate resources necessary for your specific game? These include sonically treated work spaces, equipment, software and sound effects libraries that are compatiblewith the needs of your game.
  6. Does your sound team have a demonstrated track record of producing sound within the style and genre of your project?

If you answered "yes" to all of these questions, your sound design team is properly equipped, prepared and ready for production. Answering "no" to any of these questions will tell you where you
will need to focus portions of your budget.


B. Outsourcing

If the sound team is incomplete or in any way compromised, you should consider outsourcing an appropriate amount of the workload to game audio specialists. Look for companies and people that have
a strong resume of interactive sound production, and have successfully produced sound for "high profile" titles. If your game has special stylistic needs, then consider companies that have a track
record of producing sound for similar titles.


C. Foley

The overall 'theme' of your game will help dictate where you may need additional resources. A historically based game will require authenticity; therefore consider obtaining fresh recordings of
historically accurate weapons and vehicles. If your game focuses on destruction, a sizeable Foley session may be appropriate to produce original content unencumbered by overused sound effects
libraries. A small but well organized recording session can give your game a lot of fresh spark without breaking the budget.


IX. Marketing Your Sound

Game marketing has typically focused on the creator, developer or the voice actors within the game. In many cases, sound can be used as a marketing tool as well. For "The Incredible Hulk -
Ultimate destruction" we hired some of Hollywood's finest sound recordists to coordinate a Foley session that would produce the raw destruction sounds we needed to create the sound effects necessary
for this game. Our session took place at an auto-dismantling yard in a southern California desert. A giant forklift and bulldozer were used to drop, drag and tear apart cars, vans and trailers.
Multiple video cameras captured the session for future use on the "Behind-the-scenes" reel. The added benefit was the marketability achieved by everyone's dedication to producing the most destructive
sounding game to date. Your ability to market your game's sound will also help raise any additional finances needed to bring your sound up to the next level.


Steve Kutay is the co-founder of Radius360 an award-winning audio Post Production company, specializing in sound for film and games, located in Los Angeles,
California. For more information please visit www.radius360.com or contact Steve at "mailto:steve.kutay@radius360.com">steve.kutay@radius360.com.


©2006 Steve Kutay, Radius360. This article may be posted in partial or entirety as long as credit is given to Steve Kutay, Radius360 and www.radius360.com








Comments

Note: Please offer only positive, constructive comments - we are looking to promote a positive atmosphere where collaboration is valued above all else.




PARTNERS