General questions about Testing

Started by
13 comments, last by frob 6 years, 3 months ago

I have general questions about Testing.

1. Do testers test in debug mode to test for the games? If not, then do testers have to test the overall game quality in just one sitting?

2. If a certain bug is hard to trigger, should testers be skilled enough to trigger said hard bugs?

3. Is there a difference between Quality Assurance and testing or are they both the same thing?

4. I heard of tales that testers are generally treated badly. Is this true or false interms of working conditions? Is it also true that testers are paid poorly?

5. Besides testing for the overall quality of the game and reporting them via use of verbal/social communication skills, what else are testers suppose to do?

6. Are programmers in general very dependent on game testers for their work in programming the overall game?

 

Advertisement
22 hours ago, Paarth said:

I have general questions about Testing.

1. Do testers test in debug mode to test for the games? If not, then do testers have to test the overall game quality in just one sitting?

2. If a certain bug is hard to trigger, should testers be skilled enough to trigger said hard bugs?

3. Is there a difference between Quality Assurance and testing or are they both the same thing?

4. I heard of tales that testers are generally treated badly. Is this true or false interms of working conditions? Is it also true that testers are paid poorly?

5. Besides testing for the overall quality of the game and reporting them via use of verbal/social communication skills, what else are testers suppose to do?

6. Are programmers in general very dependent on game testers for their work in programming the overall game?

1.a. SDETs do. Most testers just do black box testing, not white box.

1.b. No. Multiple sittings required.

2. Question assumes universal answers. Testers who manage to find the "trigger" should be commended for their tenacious perseverance.

3. There is, but it's not important.

4.a. True.

4.b. True.

5. That's not enough?

6. No.

-- Tom Sloper -- sloperama.com

14 minutes ago, Tom Sloper said:

1.b. No. Multiple sittings required.

Ok so in the case of playing AAA games. How can it be doable for testers to play/test the overall game in one sitting? Like how can testers test saving, game progressing, data checking etc separately if there is no debug mode?

14 minutes ago, Tom Sloper said:

4.b. True.

Ok so then how is a game testing job in demand if its not a well paid one? And how does it not work like that for a programmer's job?

14 minutes ago, Tom Sloper said:

5. That's not enough?

Didn't you once say in your lessons that nothing is ever enough?

 

7 hours ago, Paarth said:

How can it be doable for testers to play/test the overall game in one sitting? Like how can testers test saving, game progressing, data checking etc separately if there is no debug mode?

There are many different kinds of tests. 

One type of test is to jump around to key areas to ensure nothing is obviously broken.  This typically means starting the game, getting into gameplay, opening menus, and running through some very basic functionality. This is called a smoke test or a build verification, with the "smoke" name coming from the terms for major systems being on fire, looking for smoke or signs that the major systems are broken.  These usually take a few minutes, and is one of the first tests that happen to a new build.

It is fairly rare that testers play the entire game in one sitting, and for some games, such a continuous play-though test requires multiple people to run it continuously or requires the tester to pause for the night.  Usually a continuous playthrough only happens with smaller games that can be won within a few hours.

As for no debug mode, there are typically cheats enabled for testing. Skip to levels, skip to zone, spawn items, lock player health so no damage, kill nearby monsters, complete the round, score a point instantly, that sort of thing.  Details depend on the game. A golf game could have cheats to jump to any hole, drop the ball anywhere on the course, give an instant win or perfect shot by teleporting the ball to the hole, and other testing cheats to make it easier/faster to test arbitrary conditions.

8 hours ago, Paarth said:

how is a game testing job in demand if its not a well paid one? And how does it not work like that for a programmer's job?

Game testing generally requires a minimal skill set. Testers generally range from high-school dropouts to college students, hired on a short-term contract from 1-6 months, and dismissed at the end of the project.  Testers need to be creative at finding new ways to break a game, but most testers do little more than superficial tests.  Skilled testers tend to be hired permanently and are quickly promoted to lead roles among other testers.  

A game tester will generally earn more than less skilled jobs, more than flipping burgers or working a cash register at Walmart, but it is not highly paid.

A game programmer, on the other hand, is a highly skilled job.  Programmers generally get the highest salary of all the game development roles. There is an annual game developer salary survey put together by Gamasutra, Game Industry Career Guide, and a few other groups you may wish to read.

8 hours ago, Paarth said:

Didn't you once say in your lessons that nothing is ever enough?

Context is important.  Many things are enough, some things are never enough.  Often looking for "enough" is a person's way of asking "what is the minimum I have to do?"  Other times, never being enough is for tasks that must be continuous through your life, like constantly learning.  Still other times, games are never truly done, they reach a point of being good enough to ship, as good developers could find improvements to make for many decades.

In this case, referring to the workload of testers, that is plenty of work to keep testers busy.  As the game gets close to final and is finishing out the last few rounds of testing, it is often enough for hours of overtime work as the overtime hours are generally cheaper than hiring new testers and getting them trained on the game testing procedures.

9 hours ago, Paarth said:

Ok so in the case of playing AAA games. How can it be doable for testers to play/test the overall game in one sitting?

It isn't. Multiple sittings are required (as I said before) - and the game must be played multiple times, with different game settings, and with different test goals.  

1 hour ago, frob said:

There is an annual game developer salary survey put together by Gamasutra, Game Industry Career Guide, and a few other groups

There used to be. Sadly, those organizations haven't done one in the past couple years.

-- Tom Sloper -- sloperama.com

10 hours ago, Tom Sloper said:

5. Besides testing for the overall quality of the game and reporting them via use of verbal/social communication skills, what else are testers suppose to do? ... Didn't you once say in your lessons that nothing is ever enough?

Testing and reporting are the primary job of testers. Not sure what else you think there might be that a tester has to do.

-- Tom Sloper -- sloperama.com

I've been a QA tester for both a AAA company and several indie game companies. There's many different levels of it. There's one who test the game while it's being worked on, there are those who test it after builds have been compiled, there are those who test for certain bugs (art, gameplay, audio, etc.), it's a multi-layered thing, and it takes time.

My former AAA job had me playtesting as many aspects of the game I was assigned to every day. And every day I tried to find new bugs and errors based on the new builds that were made.

Yes, QA testers have been treated poorly in the past, and yes, the pay could be better. However, it also depends on the company itself. The one I worked for was VERY kind to me, and kept us all excited to come into work the next day. And for a college kid? The pay was good.

QA Testers are a vital part of game development. They help make things better so that the gamers aren't the ones finding it. 

Ok so just one last question.

If a tester fails to see a bug that he/she did not notice in the end, should the tester be blamed for that?

 

Generally no. Studios hire many testers so there are many people constantly viewing the game hunting for issues. What one person misses another person probably notices. Even with many testers, games can have bugs that go unnoticed or unreported for months, and games ship with bugs that were not discovered during testing.

As one example, in a long-running franchise I was in, at the end of each round a cash tally was shown and added to the player's accumulated in-game wealth.  A tester decided to record the funds before the round, record the cash that was shown added during the round, and discovered that it did not equal the amounts shown at the end-of-round tally. More in-game money was coming from somewhere but not reported on screen or in the wrapup screen. No idea how long the bug was in place, but it shipped on at least two prior years and no testers caught it, nobody reported it online.  

Ok so I am currently been doing testing where I strain like 8 hours a day testing about 50 mini games. Thing is, its not very effective or even efficient because I'm doing testing without any testing tools or any debug tools to make testing easier for me. When some bugs occur, programmers and the producer and the manager that I'm stuck with are blaming at me for not checking the games properly even though they know that one tester isn't adequate to get the job done.

I'm trying to lecture the programmers to give a debug mode to make the testing process easier so that things can go as smooth as possible but they keep refusing or say that doing that is difficult for them. 

The other issue is that whenever I finish the thing on time, these programmers and the producer and manager all of a sudden give me a build and force me to check it before I leave kinda like a blackmail. Everytime I report bugs, they are not updated on the report because these programmers don't check their mails and their reports as often as they should for some reason.

It also doesn't help that even basic discipline isn't working here as well because nobody shows up to office on time. The office time is 10:00 AM to 7:00 PM but these programmers and others come as late as 12:00 AM or 1:00 PM and I have to often sit with them working extra.

I get no respect around me, whatever advice I give to make things more effective are not being practised and I'm trying to be as lenient as possible but Its just not working out as it should. Its a startup company and I get that but at some point, the company has to end up doing things effectively and efficiently like adding debug modes.

The games that I have tested have three modes in them and all have to be played to ensure quality. All I have done is to check if it is playable at the very least because fixing minor bugs isn't possible due to interest of time or that they don't feel like doing such minor things.

Because I live with my parents, Its not possible for me to do double shifts or even work extremely late like 12 night or something because if that happens often, I have no choice but to quit the job and move elsewhere. 

So what can I tell the programmers so that they can avoid blaming me often and just make it so that we can both work effectively and efficiently?

This topic is closed to new replies.

Advertisement