Josh,

I attended a talk about this very subject. I don't have my notes handy, but I recall these pointers:

- The designer should not be present and visible to the test volunteers. If the test volunteers know they're talking to the game designer, they can soften their remarks. Ideally, the designer could be in a nearby room, monitoring by camera and microphone. It's best if the person running the test session be an uninvolved party so as to elicit the most honest feedback.

- The questions to be asked of the test volunteers should be very focused. Vague questions get vague responses. Know exactly which features you want feedback on, and focus the questions on those.

- Record the test.

- Ask questions before the test, and ask more questions after the test. Let the test volunteers know what you're going to ask them after they've played the game.

- While watching the test, new questions can occur, and those can be passed along to the person running the test.

Good luck with your test.

Indeed, what Tom Sloper said.

You may want to study psychology and social methods for studying, for example your test subjects can't receive help from you AT ALL. If they fail to install the game, you can't help them. If they fail to open the game, you can't help them. If they get stuck on a level, don't interrupt them to give tips. Even if they ask you and insist, you have to say "it's up to you" (but record the question, it's very likely they're confused by something that needs to be addressed). Even if they failed to do something as simple as installing the program or get to the first level from the main menu, there's a 95% chance it's your fault and has to be fixed.

Additionally, you may want to include statistics gathering in your game. For example FPS games record areas where players get killed more often; you can record average life meter, length to pass each level, areas where player spend most/less time, deaths per match/level/etc. Whatever applies to your game.

For example if they spend too much in an area, it may be because it's poorly balanced (an area having too much to explore), players get stuck on something, it's a hard level, its layout looks like there's a hidden/hard to reach area (even though there's none; aka it looks more interesting than it should be), etc, etc... you get the idea.

If you haven't already, I recommend checking out the

Great recommendation. One point worth noting: in the video the cartoonist shows the survey question: "rate this game." Don't ask that. Ask "would you buy it" or "how much would you pay" or "would you recommend to your friend/kid brother/grandmother" instead. That's what I meant by "focused" questions. Also: "rate the controls," "rate the graphics," "rate the story" -- all more focused (thus better) than "rate the game." The cartoonist is just using shorthand (it's not a flaw in the video).

First you need to ask a question to yourself, what exatly you want to test? For example testing for bugs is a total waste of testingsession since this can info be gathered easily online by simple "post here all bugs" topic in your forum.

My favourite is to get a person who never played my game and put in front of a computer in me behind his/her backs. I use it for testing interface, confusion and first time experience (the most valuable and hardest to get info). Then I shut up and watch, sometimes I make soft remarks how "unsmart" the player is (which assures I will get a honest opinion without holding anything back due to their politeness at the end :D). I note how much the player cursed/hesitated/got lost on each aprt of the game. At the end I ask "what you think" and I get a list of graviences, sometimes I also ask "what was confusing" (a very good question, since there is no valuating the game, they are telling about their experience not if the game was good or bad, it's comfortable to people). The worst question in my book would be "how you rate the game", very stressful and unlikely to get a honest answer, plus it does not give you much/any info.

Generally, the key is to use the time properly, don't waste it for pointless things you can get by other means. No speeches, no showing documentation, no explaining what the game is about (or, God forbid, introducing the story), except for a very short two line introduction.

Do not forget that playtest session is not to get testing, it is to get specific kind of testing. There are other (much cheaper and easier) ways of testing that should be used for other things.