Sign in to follow this  
noodleBowl

What to do for debugging

Recommended Posts

A few days ago, I wanted to test some initial systems I made and see how they would perform on a computer that wasn't my dev pc.

 

So I handed out some EXEs to my friends, first friend everything runs perfectly. Second friend instant crash! I thought maybe it has something to do with graphics drivers, so they tested it on a different computer... Boom crash again! Having no idea why I added more debug and checkpoint statements. After a hour or so I finally had enough statements to pin point the problem, figured out I never set a GLEW boolean to true

 

Then this got me thinking about debugging. What happens if something like this happens again? I don't want to go back and add a ton of statements every time a problem happens. Normally I would add some global debug variable to switch debugging on/off, but when I think about it that can't be that great of a solution right? I mean, you'ed have a if check for every block of debug statements. That's got to hurt performance to do all those checks, especially if you have a lot

 

So I'm wondering what do you guys do? Or am I doing it?

 

 

Share this post


Link to post
Share on other sites

The overhead of checking your global flag isn't that great, provided you don't do it too often.

 

But don't make the mistake of doing other expensive processing, even if the debug flag is off. Don't format complicated debug strings, then pass them to a debug function to be ignored. Instead, do the check first, before the formatting.

 

---

 

You might also want to consider enabling post-mortem debugging in your app. How to do this depends on the environment, but it's usually possible to collect some kind of crash dump, and ask the users to send this back, which lets you see what went wrong in a production environment that you can't necessarily replicate. A lot of (e.g. PC) games do this.

Share this post


Link to post
Share on other sites

I use log files and crash dumps.

 

Log Files

Very good when starting the game as there is often problems there with drivers and everything. I prefer to log each subsystem. Example.

Init Keyboard - Done
Init Audio - Done
Init OpenGl - 

Gives a good hint that something is off with the opengl system as it died before the Done log message smile.png. While the game is running i avoid logging anything but major events such as changed level and return to main menu.

 

Crashdumps

When everything crash and burn it's nice to have a crashdump to fall back on. As SeanMiddleditch said you need to keep the pdb's for each version you release and setup some sane way of tagging them with version numbers. Everyone will swear by their mothers grave that they used the latest build and they will be wrong so it's best to have a version number in the crashdump filename dry.png . When you open a crashdump (in visual studio) you get the callstack of the crash and some of the variables used in the call. Not a direct answer but it makes it a bit more easy to guess what is wrong cool.png .

 

I have an example of a crash system i have done. Really old code for visual studio 2008 and not so sure it works anymore but you can always have a look and get some ideas. Check it out at http://spinningcubes.com/?page_id=958. That system is made to include the pdb's in the build so artists and other developer can get direct crash messages. But it should also be possible to toggle on so it writes down a minidump only.

Edited by DUDVim

Share this post


Link to post
Share on other sites
As for knowing when to log it, remember to check every return value.

Many people, beginners especially, will pull code from tutorials and guides where the code is meant to be an example with everything going right.

Whenever your function returns a status code you should check and verify it is correct. If incorrect, drop a log entry and handle the response.

Some people are afraid of all the if(status!=0) blocks after each major command, and if you're among them you can use the likely/unlikely flags on GCC or PGO optimizations

Share this post


Link to post
Share on other sites

It's best to collect crash dumps from executables you distribute. You can then load these crash dumps with the PDBs generated _at the same time you built the executable_ into Visual Studio quite easily (meaning that you must keep your PDBs around for any executable - or DLL - that you distribute).

The problem with adding logging is that you very probably didn't add all the logging you need to debug the problem. Then you're having to continuously deliver updated executables to your long-suffering users, who are continously sending you slightly different log outputs in the hope that the problem is fixed.

With a crash dump, you can immediately inspect the back trace and see exactly where the crash occurred, inspect the stack (to a degree, anyway, since it _is_ a Release mode executable) and even some parts of the program's memory. Basically, you can load up the crash right into your debugger just as if the crash had happened on your own machine, with a few limitations.


I definitely want to add crash dumps to my system to make my future testing easier.
With C++ I'm actually really fuzzy when it comes exceptions/crashes. I pretty much have no experience in this area sad.png
 
I understand what crash dumps are and I know that PDB files are generated with my build (my IDE is visual studio 2013).
But I still do not understand to actual use them. Also what if I want to run my application on OSX or Linux, will those PDB files work?
When I think about it I'm not actually sure how to actually generate a crash dump wacko.png 
 
I mean it can't be as easy as:

 
void main
{
     try
     {
          /*
           Normal game code. Everything. Update Loop, Render loop, Input code, etc
          */
     }
     catch(exception &e)
     {
           /*Generate Crash dump*/
          /* Create a new file and write all the exception things */
     }
}
 

Share this post


Link to post
Share on other sites


I understand what crash dumps are and I know that PDB files are generated with my build (my IDE is visual studio 2013).
But I still do not understand to actual use them. Also what if I want to run my application on OSX or Linux, will those PDB files work?
When I think about it I'm not actually sure how to actually generate a crash dump

It won't work in OSX or Linux because they don't use Windows executables.  (Well, they can if you are using WINE, but it still works the same.)

 

It is almost as easy as you wrote.

 

Create your exception handler with the signature: LONG WINAPI MyCrashHandler(EXCEPTION_POINTERS * ExceptionInfo);

 

You can put whatever popups and useful information you want, but make sure it calls MiniDumpWriteDump().  That function will write your minidump file.

 

Then in your main function, register your function with ::SetUnhandledExceptionFilter(MyCrashHandler);

 

That will capture most stuff, but there are a few situations where the system terminates your program without calling it, like fatally corrupting your heap.

Share this post


Link to post
Share on other sites


That will capture most stuff, but there are a few situations where the system terminates your program without calling it, like fatally corrupting your heap.

I believe I mentioned this in another thread, but we usually set up a separate launcher that starts the game and attaches itself as a debugger so it can catch the exceptions and write the mini dumps. It's a little more work to get up and running, but being able to catch those extra tricky bugs where the game itself is indeed in a bad state, is totally worth it. Plus you prevent someone from attaching their own debugger to the game, which isn't a huge deterrent for a determined hacker but it's low-hanging fruit that comes for free with the crash dump collector :)

Share this post


Link to post
Share on other sites

I believe I mentioned this in another thread, but we usually set up a separate launcher that starts the game and attaches itself as a debugger so it can catch the exceptions and write the mini dumps. It's a little more work to get up and running, but being able to catch those extra tricky bugs where the game itself is indeed in a bad state, is totally worth it. Plus you prevent someone from attaching their own debugger to the game, which isn't a huge deterrent for a determined hacker but it's low-hanging fruit that comes for free with the crash dump collector smile.png


But doesn't attaching a debugger do a hit on your game's performance? Also wouldn't the end user need to have the debugger installed. I think I'm misinterpreting you and your talking about a custom watchdog like process. Even still how does it know if the game crashed?
 

It won't work in OSX or Linux because they don't use Windows executables.  (Well, they can if you are using WINE, but it still works the same.)
 
It is almost as easy as you wrote.
 
Create your exception handler with the signature: LONG WINAPI MyCrashHandler(EXCEPTION_POINTERS * ExceptionInfo);
 
You can put whatever popups and useful information you want, but make sure it calls MiniDumpWriteDump().  That function will write your minidump file.
 
Then in your main function, register your function with ::SetUnhandledExceptionFilter(MyCrashHandler);
 
That will capture most stuff, but there are a few situations where the system terminates your program without calling it, like fatally corrupting your heap.


Linux and OSX have their own equivalents right? Or on Linux and OSX you need a custom dump?

 

Would it be better to write my own custom crash dump? Where it does not use OS API methods like MiniDumpWriteDump.

Something that catches exceptions like in my above post?

Or would that be a "bad way" to handle things, becuase itsn't that what the MiniDumpWriteDump and SetUnhandledExceptionFilter calls basically do?

Share this post


Link to post
Share on other sites


But doesn't attaching a debugger do a hit on your game's performance? Also wouldn't the end user need to have the debugger installed. I think I'm misinterpreting you and your talking about a custom watchdog like process. Even still how does it know if the game crashed?

If there are any performance issues, they're probably negligible compared to those of the game. At least it's never been an issue for us.

 

And you can debug a another process by using the DEBUG_PROCESS flag with CreateProcess. Then you will receive debug events and exceptions from that process and can handle them accordingly. The user doesn't need to have any special debugger installed, all that functionality is built right into Windows.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this