Jump to content
Sign in to follow this  
  • entries
    2
  • comments
    0
  • views
    524

The Art of Bug Reporting

iXieGaming

1154 views

Art of Bug

Introduction

Finding a bug has always meant that half the work is done. Validating the bug with the developers makes or breaks the effort taken in finding the bug. Usually, a lot of bugs are overlooked due to the fact that the explanation has been poor/incomplete/ under-defined. At the other end of the spectrum, the bug’s explanation has been extended to the point of exhaustion. The following blog will break down the factors that need to be considered while writing any bug.

 

Background

Since my entire career has spanned around the video game industry, I will be citing examples based on my experience. I have been in the industry for over 14 years and have witnessed a lot of trends changing in the approach. This may not be the most modern in comparison to the available technology and reporting tools. But this is the core learning that every tester should have and I feel that it is an important skillset to upgrade with the testing knowledge.

Any tester who has his core concepts set right in the testing industry will have the potential to excel in any form of QA. While the topic is very simple, mastering it is a challenge that is faced by many. This blog will not just explore into the base rules of writing a bug but will also help to map the tester’s thought process. This will help them articulate and convey what they want to say in real world scenarios.

Problems

Following are the problems / overlooked issues while writing a bug:

  • Lack of game knowledge
  • Usage of phrases/terminology
  • Condense steps to reproduce
  • Proofreading
  • Review of bugs
  • Severity versus priority
  • Easier method of reproducing the bug
  • Summary
  • Reproduction rate

Solution

Research on documentation  

 

This is probably the most important aspect before testing the game. The game documentation will contain names of objects used in the game world. Knowledge of these names creates the world of difference between the explanations being vague / research-on-documentationcommonly worded to specific instructions which are to the point. This does not just help to explain the bug but to show the developers that you are aware of the game and know all the intricate details while writing a bug. Most of the games also have zone markingsin each level which is usually not shown in-game. These markings on the documentation will help shorten the bug and be very specific in the description.

Example: the sentence “Level 18: After crossing the barricade of cars, observe the abandoned bus near the wall. If the bus is to the left-hand side, continue straight till you find the 4th house to the right. This is the gray house with the broken window on the first floor. Enter this house through the back, head to the living room and observe the missing texture above the fireplace.” Can be rewritten as “Level 18: Zone 3, House no 18. Enter the back door and head to the living room. Observe the missing texture above the fireplace.”

Also, with bug writing, it’s helpful to use the consistency of words while describing a bug. Words like user/player, input names, character names, game specific objects, weapons and other game factors need to follow the same game terminology. This consistency will help to describe the game better.

Usage of phrases/terminology – A lot of games use real-world objects. Even the game environment settings are taken from famous buildings/areas and some games even are set in famous cities. A back research of the place where the game is set and the culture research will help to not just test the game but also give inputs on things which do not align with the game.

The areas of research which can help in identifying the culture are:

  • Game environment – City/Famous places etc
  • The involved city’s background
  • The people’s way of life
  • Art/Music /Entertainment
  • Climate
  • Basic flora/fauna
  • Social festival/events

Steps to reproduce

The best way to write a bug is to write the STR before writing the rest of the bug. By focusing on the STR at first, the bug is easily divided into sections. This will help to break down the bug and explain it better.

The best way to write a bug is to have the STR condensed to 5 basic steps. This helps in keeping the main explanation of the bug focused and removes all unnecessary information. To make this easier, I have broken down how a typical bug needs to be described:

  • Step 1: Game mode (Single player, multiplayer, campaign, mini-game etc.)
  • Step 2: Which area/level/map
  • Step 3: Which location?
  • Step 4: Steps were taken to create the bug
  • Step 5: What is the bug

I find this practice not just useful while writing a bug but by putting it in any real life explanation, I find it very easy to explain multiple varieties of things. It helps you keep your mind object oriented and focus on the end issue you want to convey.

In the above 5 step analogy, there will be issues where step 3 and 4 may need a further breakdown. This is not a hard and fast rule but a basic interpretation of what needs to be conveyed. While there can be more or lesser steps, this is the optimal approach to writing a bug. This method will remove the unnecessary information and keep the focus on the main issue.

It is human nature to drift away when exposed to a long explanation. Keeping it short will definitely retain the focus and convey the point. With these points in mind, the maximum acceptable STR needs to be within 8 steps.

Proofreading

Gaming Bug ReportingThe best way to write a bug is with Word or any other formatting tool for documents. They not only help you with spelling errors but also assist you with grammatical mistakes.

One area which has been a concern with a writing of bugs is the usage of tenses. Many, who write bugs, do mess up when it comes to writing a bug while explaining the tense. The below link is a small example of how the sentence needs to be structured.

http://www.english-for-students.com/Tenses.html

Bug review

bug-reviewThis is a personal growth progress. Testers need to keep a track on how many bugs they reported, the severity of the bug and the days when they reported it. This helps to track a pattern of their areas covered, the maximum productivity of the week and the amount of contribution they have to the project.

One major thing that they need to track is a number of bugs they reported versus a number of bugs that returned with ‘need more information’ or ‘moot’. This will help to track the quality of bugs reported. This is important in a tester life as it reveals two important things:

  • The bug did not make sense
  • The tester did not understand if it was a bug or a feature

The second part is a bigger worry because the tester has not had the knowledge transfer and hence, he/she are not empowered with the game. As testers, we are expected to be masters of the game and the tools we work with. Any issue in this area shows weakness and does not instill confidence of the developers.

Severity versus Priority

Severity-versus-PriorityThis has to be the most challenging debate with testers and developers. An A class bug need not be fixed even if it feels important. Now suppose the player entered a car, drove it to a port, took a boat, hooked the character to a plane, jumped off and stole a bike, then ran into the wall which crashed the game. This bug is an A class in severity. But, consider how many gamers will attempt the same stunt while reproducing this bug. This will be less than 8% of the gamers who find this issue by mistake. If they reload the game and they’re saving progress is still intact, they don’t even bother about this issue. Consider all the games that have a strict schedule or timeline. When the game is about to release, an issue with this probability is low, will be overlooked. On the other end of the spectrum, if the player chooses the left side of the bridge to cross, the screen goes blank for 5 seconds, there is still a 50% probability that gamers will face it. This is not an A class issue, but since the probability of gamers facing that issue is still high. This issue will feel important to the end user’s overall game experience.

Hence the priority of the issue does take a front step. The entire development team does focus on releasing the product on the issued date while maintaining the maximum quality. We need to respect the fact that everyone works on a certain timeline. As a QA member, we need to focus on the issues which will change end user experience. This does not mean a tester does not report all edge case issues. We as testers have our primary fulfillment to report any issue that we encounter.

Alternate bug repro steps

alternate-bug-reproWhile we find a bug, it’s not necessarily the best way to find the bug. We may be testing some other function in the game while we encounter the bug. The tester needs to be intelligent enough of the end cause of the bug and then retrace the steps that actually caused it. In this method, we encounter several other possibilities which caused this issue. We can take the simplest form to reproduce the bug. This will also teach us the root cause of the bug.

Example: On a calculator app, if I did 2*5, it crashed. While this scenario might be correct, it may fix the bug only for this calculation. But had I done 10+0, 20-10, 200-190, 20/2, etc, I would have found the same issue. The end result proving that the app has an issue with 10 being the result rather than 2*5 being the error. Hence, research into the issue and come up with alternatives. This will always help to narrow down the issue.

Summary

This is vital not just to explain the bug in a short form but it helps other testers to check if a bug is logged. Most database search strings involve looking at summary first. If the summary is optimized, the tester will spend lesser time searching for an issue instead of logging one. The summary needs to be crisp, precise and hold all important information within a sentence.

While writing a summary, know that the issue always takes precedence over other factors like location, game mode, user level etc. This will help to narrow down the issue and create a referral point for any other tester who wants to look up the issue.

Reproduction rate

This has been ignored by most testers. The above point of having alternate repro rate is vital to me having to emphasize this. Just because you find an issue doesn’t mean it’s down to the code or art. There are several other factors involved. There can be issues with the hardware or any software in the background which is causing interference. This may not seem like the bug you wished for but it is worth its weight 10 times in gold. Any background app/hardware which causes an issue to the running game, the developers spend much more time fixing this because they wouldn’t want the user to be deprived of any other app to accommodate the game. This can be a big deal since most users will blame the game and leave it to accommodate the app instead. This can be a huge negative publicity to the game and most developers tend to avoid being in this state.

As this blog tends to dwell on the issues that need to be addressed while writing a bug, it isn’t necessarily a bible to enforce how a bug is written. The tester has full control on what needs to be conveyed in the bug. All this article wishes to cover is the fact that a bug can be so much more powerful when carrying the right message. Do follow these steps and believe in it for it can be a useful method to showcase the thought process. It will not just help you in writing a bug but also assist you in formulating your thoughts.

 


0 Comments


Recommended Comments

There are no comments to display.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Advertisement
  • Advertisement
  • Blog Entries

  • Similar Content

    • By Jordan Winslow
      Hey guys, my name is Jordan Winslow and I am a professional electronic music producer & composer who also happens to be a talented story writer and has a ton of experience with VNMaker, Tyrannobuilder, Renpy, and RPG Maker tools and I am looking for talented artists who want to make a horror game together!
      Last Horror Project I Composed Music For: 
       
      My last game I created: "The Watchers"  https://jordanwinslow.itch.io/the-watchers
      My Music: https://jordanwinslow.me/showcase
      I am open to plot ideas but, based on your artwork, I would like to create an original story that matches our music and art so we can play on both of our strengths. Up until now I have only been able to create visual novels with the use of stock photography and stock videos, but with the use of original art, we should be able to come up with a story that is far more specific to the art on screen.
      My favorite horror games (For reference)
      Saya no Uta (Horror Visual Novel Originally in Japanese)
      The Crooked Man 
      SOMA
      Corpse Party
       
      I have a few pretty awesome ideas for sci-fi or extra-dimensional horror games, I also know of a great real-life story about aliens we could make a game about and I could easily come up with something new if I am inspired by your art!
       
      So let's create something awesome together! Send me a message along with some examples of your art and we will either create a team of multiple people together or just the two of us if you are capable of coming up with a decent amount of original artwork! I've got the music, the story and the programming handled unless you want to do the programming and help with the story.
    • By Sergio Ronchetti
      Continuing to work on “Eldest Souls” (first article here!), I’ve begun familiarising myself with the workflow between Fmod and Unity, and the integration system. I know much of this will be pretty obvious to most, but I thought I’d share my thoughts as a complete beginner learning the ropes of sound designing. 
      The library of sounds that Fmod provides has been very useful, at least as reference points. I’ve still kept to my ethos of producing the sounds myself as much as possible. Having said that, Fmod gives you 50 free sounds with your download, and I’ve used a wooden crate smash, a drawbridge and electricity sound you can hear in the foley video below.
       
       
      The thing i found most useful was witnessing changes i made in Fmod being realised instantly in Unity. If a volume needed changing, or the timing of one of my effects was off, i can literally switch to Fmod and then back to Unity and immediately see the result of my alterations. It also seems apparent that using middleware such as this (or i've heard Wwise is also equally intuitive) grants the developer, and myself included, a great deal more flexibility and opportunity to edit sounds without going all the way back to a DAW, and bouncing down again. Needless to say, my workflow is so much faster because of it.
      I've also loved the randomised feature of Fmod, whereby any sound can be made to sound slightly different each time it is heard. Taking a footstep recording i made for example, I was able to add further authenticity of uneven footsteps by randomising the pitch and volume of each playback. 
       

       
      I used this technique when creating footsteps for the first major boss in the game called "The Guardian". A big, over-encumbered husk of a monster. I also had fun rummaging through the garage for old tools and metal components for the “Guardian” (the first boss) footsteps. See below!
       
       
      I also created a sword attack for our player, trying to sound different from the generic “woosh” I see in so many video games. I used a very “sharp” and abrasive sound to differentiate him from any enemies.
       
       
      On another note, I recently upgraded my microphone to a Rode NTG2 shotgun, which has been phenomenal. I haven’t had to worry about noise interfering with the clarity of my objects, whereas before with the sm58 I had to be clever with my EQ and noise reduction plugins.
      Important to note again that this still a “cheap” mic in comparison to most other products on the market, and all in all my entire setup is still very simple and affordable which I’m quite proud of. I’ve seen many musicians spend heaps of money on gear they don’t necessarily need. I much prefer being resourceful with less equipment, than to have more than I can understand or remember how to use.
      It’s forced me to understand every aspect and capability of my tools, which I believe is a principal that can be applied to any discipline.
       
      I have more fun little sound effect videos on my Instagram for those interested, where I post regular updates. Thanks for reading! (if you’ve made it this far)
       
      www.sergioronchetti.com
      INSTAGRAM
      fallenflagstudio.com
    • By Sergio Ronchetti
      BASICS IN SOUND DESIGNING FOR VIDEO GAMES
       
      Recently I joined the talented team at Fallen Flag Studio as the composer for their latest release "Eldest Souls" which consequently lead me into a field I have always dreamt of trying - sound design!
      Having no prior experience, I began watching a few online tutorials (if you want to learn from anyone make it Akash Thakkar from "Hyper Light Drifter"... what a guy!) and basically just testing stuff out i found around the house. Luckily my dad has a garage FULL of random crap to use.
      Before i continue, it's important to note that i DO NOT have fancy equipment, meaning anyone can try this. (my equipment is an sm58, focusrite scarlett interface and Logic Pro X plugins... that's it!)
      I started basic with some footsteps, which weren't all too difficult. Then I moved on to projectiles and a spear attack one of the bosses has. Below are a couple super short videos on my resulting attempts.
       
       
      Amazing how great a banjo sounds for that typical "woosh" sound! And if you're wondering, the paper was added to give some texture to the jab.
      I could be finding a lot of these sounds in libraries online (like the built-in ones that come with Fmod and Unity) but I've chosen not to, in order to produce authenticity and hopefully a more unique gameplay experience for players when the final product is put together.
       
      P.S. if you'd like to try the game and hear my hard work we'll be at EGX and several other conventions later this year, soon to be announced! Thanks for reading!
      www.sergioronchetti.com
      fallenflagstudio.com
       
      To those interested, there's an Alpha trailer of the game in question below.
       
       
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!