Sometimes bugs are really nasty, coming up only after several minutes of active gameplay and once you see the result, you can't find a trace for the reason. Over the years I've added some useful features to help me finding bugs and I thought it is good moment to present them here. Larger projects will most likely utilize these and many others features, still you might find one or two ideas useful to tweak your own debugging tools for your game.
Let's start.
Log files
Okay, this is one of the most basic debug features available and a must have.
HTML Server
I think that your game should be a HTML server, it is not unusal, but nevertheless extremely powerful. For coding a server like this, you really just need a tcp socket and some basic http handling. HTTP is no magic and is really easy to parse and write. If you write out some simple html pages as responce to a http request, then your browser will suddenly be a very powerful debugging tool.
More advanced features are the usage of javascript/ajax etc. to transform the browser into a extremly powerful interactive tool.
Shader Reload
Reloading shaders on the fly (per html server plugin) and logging out some data (does it compile etc.) is really useful to track down this little nasty shader glitches.
Custom Script Execution
If your engine supports a scripting language, add a possiblity to execute script-code on-the-fly. Very useful to log out information, or to modify game objects.
Ingame Object Information
Add some way to have a ingame information about objects. Use the cursor/mouse to target an object and push out the data directly on the screen. I found it really useful to have two of this debugging outputs, one on the left side of the screen and one on the other side. This way I can track down two objects concurrently. Although helpful is the use of the page up/down keys to display multiple pages of information.
Lock on a Single Object
I can take a single object and put a debug focus on it. This results in additional debug output (logfile) etc and to in the option to set conditional breakpoints to isolate a single object in your debugger. If you debug every object all the time, then you will have just too much information to handle efficiently.
Custom Visualisation
Just add some visualization to else invisible information. From a simple wireframe render output to displaying physics hullobjects and waypoints.
Persistent Gameflow History
It is really useful to have a secondary logging mechanism to log and view the gameflow. This is similar to the standard log file with the difference, that it is part of the game (and will be saved with it), is more compact, describes only game related stuff (no technical stuff). It helps to track down these game design bugs, eg. if a tester said, that he never found the red-key, just look up the history file to get a hint (eg. he picked it up and dropped it later on). It helps to keep track of all these events you can't watch all the time (Why are suddenly half of my minions dead ?).
Time slow/forward
Ok, if your game is realtime, you should consider to use a virtual clock. I have the option to speed-hack my own engine by slowing/speeding up the time. It is really useful to reproduce some time dependent events (7x forward and the bug will be reproducable in a few seconds), to slow it down (why are this animation transition so choppy) or to even stop the time completely
to freeze your game state and debug it with your html plugins and script execution.
One thought, your forward will most likely get CPU bound due to fix-your-timestep parts of your code. In this case additonal CPU power is really useful.
That's a good summary, thanks. The HTML server stuff is interesting. Care to share some of the implementation of that?