Back to General and Gameplay Programming

What data do you log when gamers test your game?

General and Gameplay Programming Programming

Started by SpaceTraders October 15, 2014 02:02 PM

2 comments, last by SpaceTraders 9 years, 6 months ago

SpaceTraders

150

Author

October 15, 2014 02:02 PM

Hello fellow developers.

I was wondering what information would be good to log when you let people test your game?

I am using OpenGL and C#/.NET for my game.

Thanks for any replies/help

My current test-scenario is the following

1. Game Upstart

Before the game starts and switches to fullscreen a dialog is shown (only for the test-version) which collects information about video card, resolution, drivers+version. The user is the asked if it is okay to send this to the developer (me ). It is only done the first time the game is started or until the gamer accepts.

2. Normal Gameplay

My game is a tiled-and-turn-based-RPG so I can log all keystrokes + enemy movement. This can be used to replay the gameplay in case the bug can be found this way.

3. Runtime Exception

If a .NET exception occurs it shows a dialog that will ask if it is okay to send the recorded gameplay with the error message to the developer. After this the game is exited.

4. Game Exit

If no error occured during gameplay a dialog will be shown that will ask if the gamer encountered any errors. If yes then it will ask if it is okay to send the recorded gameplay to the developer and ask for a short explanation of the error.

Buckeye

10,754

October 15, 2014 04:00 PM

You don't mention it explicitly, but you should also inform the user what will be logged, and provide options for where to store the log file, and a maximum size for the file. Some time ago, I tested an app which created a huge log file somewhere, and it wasn't deleted when the app was closed. It only took me just a minute to find the #$% file and delete it, but it left a bad taste in my mouth.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.

You don't forget how to play when you grow old; you grow old when you forget how to play.

SeanMiddleditch

17,596

October 15, 2014 06:05 PM

Log _everything_. There's no such thing as too much information. You can pare it down when you need to with any half-decent data mining tool.

Log average and worst case framerates.

Log time spent in various dialogs and areas.

Log deaths, locations, and causes.

Log battle length and stats.

You don't need to record the whole game, but being able to grab a snapshot when a bug is detected or anything needed to e.g. reproduce a battle (if you're a Japanese-style rpg, this might just be character state, inventory, and the PRNG state used to select the enemies; western-style RPGs would have to snapshot more world state). You don't need a recording of the player spending 10 minutes walking around town, but you _do_ want the fact that the user spent 10 minutes in the town (which helps you find out if the player got lost/stuck).

Remember, asking playtesters for feedback is one of the worst ways to actually get it (the only worse way being to not proactively collect any feedback at all). Measure everything. If players keep dying on level 3, they're unlikely to outright tell you what's wrong; measurements will help you see that 50% of your player base died there and from what, which then tells you with no uncertainty that there's a difficulty spike problem there).

I also highly recommend in-game error reporting buttons or keys. Don't ask the user after the game if they ran into errors; they might have forgotten some of them or just be far too lazy to type in an explanation (this is true of all of us). You have to eliminate every possible barrier to reporting issues as is possible. Make sure there's a big red Report Bug button in the corner of the screen at all times. If it's clicked, then send in a snapshot of the state and playback information from the last minute or two (a twitch stream is almost as useful as an in-game playback; you often just need to see what happened).

Sean Middleditch – Game Systems Engineer – Join my team!

SpaceTraders

150

Author

October 16, 2014 07:34 AM

Thank you both for your replies. Your inputs are very helpful.

Buckeye

You are right that the playtester should known exactly what is logged so that he/she is willing to participate in the testing. I am going to store the log-file in a subdirectory of the game. Good idea that the playtester should be able to the delete log-file(s) if/when it is needed.

SeanMiddleditch

My game is very similar to Ultima I Just with a bit better graphics (keeping it old school) and hopefully an engaging story. I am using the "http://www.dungeon-world.com/" as a rule set. I have implemented a c#-DLL that includes most of the rules and attributes. I am thinking of open sourcing the DLL at some stage so that others can use the "dungeon world" as a base for their rpg games.

Logging everything could be a lot of data and I am afraid of an information overload. But then again I was only think of handling bugs/errors but you are right that logging how the game is played would tell me if some corners of the game is not used/or too difficult etc.

I think you are right about that playtesters might not be too good at reporting problems after they exit the game.

In-game error report with a screenshot taken is a really good idea.

What data do you log when gamers test your game?

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

What data do you log when gamers test your game?

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Reticulating splines