• Announcements

    • khawk

      Download the Game Design and Indie Game Marketing Freebook   07/19/17

      GameDev.net and CRC Press have teamed up to bring a free ebook of content curated from top titles published by CRC Press. The freebook, Practices of Game Design & Indie Game Marketing, includes chapters from The Art of Game Design: A Book of Lenses, A Practical Guide to Indie Game Marketing, and An Architectural Approach to Level Design. The GameDev.net FreeBook is relevant to game designers, developers, and those interested in learning more about the challenges in game development. We know game development can be a tough discipline and business, so we picked several chapters from CRC Press titles that we thought would be of interest to you, the GameDev.net audience, in your journey to design, develop, and market your next game. The free ebook is available through CRC Press by clicking here. The Curated Books The Art of Game Design: A Book of Lenses, Second Edition, by Jesse Schell Presents 100+ sets of questions, or different lenses, for viewing a game’s design, encompassing diverse fields such as psychology, architecture, music, film, software engineering, theme park design, mathematics, anthropology, and more. Written by one of the world's top game designers, this book describes the deepest and most fundamental principles of game design, demonstrating how tactics used in board, card, and athletic games also work in video games. It provides practical instruction on creating world-class games that will be played again and again. View it here. A Practical Guide to Indie Game Marketing, by Joel Dreskin Marketing is an essential but too frequently overlooked or minimized component of the release plan for indie games. A Practical Guide to Indie Game Marketing provides you with the tools needed to build visibility and sell your indie games. With special focus on those developers with small budgets and limited staff and resources, this book is packed with tangible recommendations and techniques that you can put to use immediately. As a seasoned professional of the indie game arena, author Joel Dreskin gives you insight into practical, real-world experiences of marketing numerous successful games and also provides stories of the failures. View it here. An Architectural Approach to Level Design This is one of the first books to integrate architectural and spatial design theory with the field of level design. The book presents architectural techniques and theories for level designers to use in their own work. It connects architecture and level design in different ways that address the practical elements of how designers construct space and the experiential elements of how and why humans interact with this space. Throughout the text, readers learn skills for spatial layout, evoking emotion through gamespaces, and creating better levels through architectural theory. View it here. Learn more and download the ebook by clicking here. Did you know? GameDev.net and CRC Press also recently teamed up to bring GDNet+ Members up to a 20% discount on all CRC Press books. Learn more about this and other benefits here.
Sign in to follow this  
Followers 0
freyastudio

Appropriate number of bugs

18 posts in this topic

Our project Burnt Islands reminds me more and more about the project "European Air War". We have sometimes so many bugs that a huge amount of time goes to fixing them and not actually working on new features.

So, as I look at our issue tracker (Redmine) right now we have approximately 50% bugs and about 50% features and this is after fixing a whole lot of them for the past 3 days. As we like to rest and enjoy some Thursday - Saturday drinks, one of us thought that it could be motivating if we needed to fix at least 3 bugs or features in order to earn the right to get a drink!

It became a kind of competition where the one who is not working at a day job is able to do three quick bugs due to the knowledge of the code base, while the other one needed to sit for 2 hours to get a drink!

Exhausting but also a real motivation to get things done. One of the days it took so long time to figure out the fix that it was already too late to drink! Well, it resulted that complex tasks are now divided into smaller ones so that it doesn't take too much time to fix them.

Whatever works for you this what we are doing now. But is that right? How do other indie developers do things? Do you guys also have many bugs?

3-bugs-1-drink has the disadvantage that larger and more complex bugs/features aren't started after a while. Maybe it's better to work 1 - 1.5 hours on the night shift before getting a drink?

One of us works as an enterprise application programmer at her day job and here is how the big guys do things:

- all major tasks are being specified and analyzed beforehand and is then divided into smaller tasks.
- developers get those small tasks and implement them in a period of 1-3 weeks (Agile / Scrum method).
- at the end of each iteration everything that was done gets tested and bugs are being fixed on the fly.
- next iteration - new tasks.

If we were using this method for our game development then the number of bugs would grow so huge after 3 weeks that I think it could take up to 1 month just to fix them. If we used those 2-3 days at the end of the iteration and no more then the game could look pretty much like "Day one Garry's incident" game that is full of bugs and received a horrible review.

We don't want to release a bad game, so we wonder what is the best way to do things so that you implement new features and keep the bugs down?
(oh, we can't write bug-free code if anyone was wondering).

And what is the usual proportion of bugs vs new features in the game development industry?

0

Share this post


Link to post
Share on other sites

I don't have a good intuition for the ratio of bugs/new features on any project I've worked on. Typically when a new features goes into the game we make sure that the people affected by the feature (designers for gameplay stuff, artists for tools or rendering features, etc) test it out and report their bugs. Those bugs are worked out as soon as possible, and the person responsible for that new feature doesn't start doing something else until the initial bug reports are cleared up. This has a number of benefits: many bugs will be caught early on, and they can be fixed quickly while the feature is still fresh in the engineer's mind, you can be relatively confident that your new features are pretty functional, and the remaining bugs will hopefully only be edge cases. Also, when you develop like this you tend to get a sense of how long a feature will take both to implement AND test, which helps you budget your time better.

 

Another thing to think about: are you doing all you can to catch bugs quickly? "Crash early and crash often" is a motto you'll hear every now and then. Basically: if you have an opportunity to detect that something is wrong in your code, you should do so, and crash if something is wrong. For example, if you have an array class, when people index into the array you should assert that the index is within bounds. If you have a pointer that you want to dereference you should assert that it is not null. If you're working in C# those things will throw exceptions by default, but the principle still stands: Crash early and crash often.

2

Share this post


Link to post
Share on other sites

I have to imagine the design phase isn't very detailed. Are you guys just jumping right into code for new features or are you designing on paper in detail first the new feature and how it needs to connect with the existing codebase? How detailed is the overall project on paper? What's your project methodology? What kind of bugs are you seeing? If there is a very common bug that keeps coming up then you need to address that situation if possible. Categorize your bugs and learn from them.

 

You have to find out why you are getting so many bugs before you can do something about reducing them. Of course you aren't going to write bug free code, but there are a ton of things you can do to greatly reduce the amount of bugs. Are you unit testing? Have you thought about TDD? These are all in place to reduce bugs and make more robust code. You are feeling the reasons as to why these things exist. 

 

 

 

oh, we can't write bug-free code if anyone was wondering

 

By saying this you are brushing under the rug the main issue. By asking how many bugs are common in the industry you are trying to almost justify your bugs and make your team feel better about their situation. If you feel there are too many bugs then you need to find out common methodologies that exist today to help you write less buggy code. The only way to fix too many bugs is to write code with less bugs. There is no getting around that.

 

We haven't seen any of your code and I've been on a project that had the worse possible codebase possible. I mean it was a disaster in every way shape or form. Nothing was good about it. There was no thought of the main design. It was just thrown together. It literally had zero chance to make it. The team just thought as long as they got something to work at one point in time it was good enough. Be honest and ask yourself if the games code is in this shape. Maybe get outside consulting to just look over the code design to see what their thoughts are on it. They don't have to write code or even fix it but be an objective 3rd party looking at your code. Maybe have a couple people outside look at it and take their feedback to heart.

Edited by rpiller
2

Share this post


Link to post
Share on other sites

Hi there.

 

Thanks for your time. The bugs we're facing are more or less "design bugs", which is as you say, a result from a lack of a detailed design document on paper. But on the other hand, when we started this project we didn't have any idea of what game we were making. We started by making a terrain generator, where you could play around (goof around) and destroy the terrain.

 

The code is written in C++, and we're using the Entity-Component model with messages for passing information / data between the different parts of the game. The technical part of the code is robust (namespaces, library wrapper projects, not reinventing the wheel etc).

 

For some of the actual bugs we're having at the current time of writing are:

 

- No shadow on certain objects

- Some objects intersect (physics model is smaller than the graphics model)

- The game sometimes crashes hard.

- Certain objects are not loaded when loading a savegame.

- Some Components aren't cooperating that well (the small buddies, your helpers, are just goofing around when they should be gathering resources for you)

- Player has the wrong orientation when starting / loading the game

- Trees aren't unloaded when the terrain under is unloaded.

 

It's just an example of some of the bugs present.

 

We have unit tests, but it is too time consuming to implement for many of the components of the game. There are tests for the code which is simple to test (math, vector++). It's not trivial to implement tests for unloading of objects when the terrain is unloaded, and it takes time to maintain those tests when the original code changes.

 

Are there any best practices when there isn't a detailed design document explaining every detail of the game?

0

Share this post


Link to post
Share on other sites

We have unit tests, but it is too time consuming to implement for many of the components of the game.

 

I'm sure you see the irony of that statement now.

 

I would get a design doc asap. If your employees don't see the final end result it may be hard for them to really understand how to efficiently build something. Building something in isolation (which is basically what you are doing because what's next isn't known by anyone) can be hard if there isn't some master plan. Imagine building a house without a blueprint. Things wouldn't fit together very well at all and that sounds like some of the issues you have.

 

On the technical side I assume you have basically zero global variables? That's always a red flag for me and the first thing I look at.

 

Since you are using a messaging system for object communication (the heart and soul of your game) do you have a global document showing all the possible communications between all objects? If not it can get confusing as to what's actually happening as logic jumps all over the place. This document should be updated and talked about all the time as it's the way all your objects talk which in turn basically makes your entire game run the way you want it. If you don't have a map of this then you are driving blind and mistakes are easier to make. The messages have requirements/parameters and expect certain values most likely. If one person makes a message and assumes certain things and another person sends the message assuming something different then you'll get a bug. This would be a great thing to unit test. Inside each object receiving a message validate the message and it's parameters are what you would expect and the handling of the message acts appropriately, giving an error message that helps you narrow things down quickly. Error handling inside messages is often overlooked because when you write the handler you know how it should work, but when working on a project with multiple people they may not know the exact details.

0

Share this post


Link to post
Share on other sites

Good answer from DekuTree64. A lot of your bugs sound like features that were checked in without being fully finished. It can be tempting to check something in when it's only 90% complete, because you want to keep moving forward and work on something else, but you'll find (and you HAVE found) that kind of behavior adds a burden on you in the long run. When you work that way you accrue a sort of "code debt," and like all debt, it's gotta be paid off eventually.

 

Slow your pace down a bit. Test as you go. Make sure your features are working before they get submitted. You'll sleep better at night.

 

Also, bugs in your physics system are to be expected. Make no mistake: you have to stay on top of those bugs, but don't get too discouraged if you've got a lot of physics bugs. Physics and collision detection are hard. Every project I've worked on has had a lot of physics related bugs. You just have to grind through those bugs (or use a physics library, but I've never worked on a project with a licensed physics library so I can't offer much advice about that).

2

Share this post


Link to post
Share on other sites

If you think unit testing takes to long, it's because you haven't used it enough to learn how to do it right.  Once you get yourself setup with the right testing tools and get into the habit of test driven development, you will never go back.  Right now the tests you are writing are probably just all wrong, because it's not something you have really bought into and made an effort to learn to do well.

 

In my experience the secret to good testing is to adhere strongly to the single responsibility principle, and mock/stub anything that's not part of what you are actually testing.  These two things go hand in hand with writing tests that are easy to reason about, and actually valuable.

 

It will take you about twice as long to write your code when using test driven development, as a general rule.  That seems like a lot until you realize just how costly it is to not find bugs until later on.  The feedback time between when you write the code and find a bug is critical and largely determines the total cost of the bug.  The longer it takes to find, the more expensive it is.

 

With C++ it's even worse.  The best C++ developers out there have basically said that humans cannot reason about large C++ codebases, they just get too complex.  So the process of tracking down a bug where you have no testing is painful and time consuming, and it will only get worse as your project grows.

 

Also, since integration testing seems to be pretty big in the game biz, I'll point out this really good talk that also has some good info on TDD in general:

http://www.infoq.com/presentations/integration-tests-scam

 

Chris

0

Share this post


Link to post
Share on other sites

I don't think it's a good idea to mix business with pleasure - you'll end up ruining one or the other. I do like the Pavlov's dog idea and I know it does work when you just want to get things done, but if you do continue to use the beer-compensation method, I'd be careful not to make it sound like it's mandatory. Maybe some of you would rather have a coke, or a pizza instead. :)

 

If you think some of the developers are getting away too easily because they only fix minor bugs, you could task them with doing code review of other developer's code. i.e.: 3 bugs + 3 code reviews = 1 beer. That way, they would be obliged to understand the harder bugs as well,while the harder working guys will have it easier, because the minor bugs are easy to review.

 

You also mentioned that there's competition... I don't think it's so much a competition - rather sounds to me like you're allowing yourselves to speed through fixing things, just to get the tasks done. This is mostly because nobody likes fixing bugs, especially the developers with more advanced knowledge of the project - they will have the least interest in bugs, because they already know how those bugs could've been avoided in the first place. At least, I don't think a beer after work should have this sort of effect on them. :)

 

That being said, I never really liked the Agile development model (but this is just me), because it forces you to code fast, without thinking about possible bugs you might be introducing. I also don't like how it categorizes and splits work into fixed, periodic categories (or whatever they're called - 2 weeks planning, 2 weeks developing, 2 weeks testing, then repeat). Since you're not documenting much, just adding tasks, you will end up forgetting details of what was planned before you even start developing. Plus, when it comes to bugs, you really cannot plan for them. I always find it easier and faster to fix bugs as I find them, while I'm still involved in the task I'm working on, even if it's not related to my work. Instead, Agile just forces you to add a new task for the bug and move on.

0

Share this post


Link to post
Share on other sites

Obviously, bugs have varying levels of severity.    You don't want show stoppers or other inconsistencies that lower the quality of the game, but if you wait to release a perfect product, you could be waiting a while.   

 

I always put new releases on Google Play first, since it's about 90 minutes to turn around an update, as opposed to a week on iOS.   

0

Share this post


Link to post
Share on other sites


That being said, I never really liked the Agile development model (but this is just me), because it forces you to code fast, without thinking about possible bugs you might be introducing. I also don't like how it categorizes and splits work into fixed, periodic categories (or whatever they're called - 2 weeks planning, 2 weeks developing, 2 weeks testing, then repeat). Since you're not documenting much, just adding tasks, you will end up forgetting details of what was planned before you even start developing. Plus, when it comes to bugs, you really cannot plan for them. I always find it easier and faster to fix bugs as I find them, while I'm still involved in the task I'm working on, even if it's not related to my work. Instead, Agile just forces you to add a new task for the bug and move on.

 

Agile is not not cowboy coding. If you plan to implement a certain feature, you're not really done with the task until the feature works as expected (i.e. no bugs). Fixing bugs as soon as you find them is a good practice, and it doesn't contradict good agile practices. In the contrary, if you are measuring project velocity (story points per sprint or whatever your measure is) then the earlier you fix the bugs the better you'll be able to measure the "true" velocity. Otherwise you are just borrowing time from the future and adding to the hidden technological debt.

 

That said, agile may not be easy to do in practice (depending on the organizational culture), and may not be the best approach for every project. On the other hand, I've not find any negative aspects of it in my experience.

2

Share this post


Link to post
Share on other sites
Wow. 
 
I'm amazed by the quality of the answers here at Gamedev (and that we actually get a response!). Thanks!
 
I'd like to clarify some things.
 
Our team is just two persons, who work with the project after the day job. This is kind of a hobby, but a little more serious than a hobby.
 
Sometimes the team morale isn't that good. I guess having ownership is a double edged sword. I don't have anyone except myself looking over how I'm performing, and some days are better than others. Some days I really get into the flow, other days I'm just to distracted / tired / sleepy / fogheaded to do anything good.
 
Most observations above are correct. I like to start new and cool features before I get to completely finish old features. Again, this is on the edge of the sword again. Sometimes having a partially working feature makes it easier to see if a feature is worth keeping or throwing away. And sometimes it's easier to come up with ideas and features when there is something (partially) playable.
1

Share this post


Link to post
Share on other sites


- The game sometimes crashes hard.

 

This is a very worrisome bug. If the game suddenly crashes at unexpected times, it could mean deep problems that could be very hard to fix, and if you release/sell this game and it crashes on people, your reputation will take a dive. By not fixing it immediately, you may have built code that relies on code that needs to be removed or was poorly designed. I personally try to keep bugs to 0 and if a bug creeps up, I fix it before moving on to anything else. When building new features on buggy code bases, future bug fixes can break working code. I've been in that situation before and found it better to stop adding features and just fix everything first. In the long run, you'll save time.

 

Buggy code can kill motivation. If you remove all the bugs before adding more features, your nicely working program will provide more motivation to finish it. "Hacky" code and endless bug fixing can drive a person insane and kill an otherwise good project.

 

I hope you fix all of your bugs first. Then you'll see how much easier and enjoyable coding your project can be.

1

Share this post


Link to post
Share on other sites


Our team is just two persons, who work with the project after the day job. This is kind of a hobby, but a little more serious than a hobby.

 

I understand when a huge mismanaged project with programmers fighting deadlines and bureaucracy and apathy has bugs pile up, but I can't imagine why a two-person hobby team would let a bug database grow out of control. That's not an excuse for more bugs. It's a reason you should have fewer bugs.

0

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0