• Announcements

    • khawk

      Download the Game Design and Indie Game Marketing Freebook   07/19/17

      GameDev.net and CRC Press have teamed up to bring a free ebook of content curated from top titles published by CRC Press. The freebook, Practices of Game Design & Indie Game Marketing, includes chapters from The Art of Game Design: A Book of Lenses, A Practical Guide to Indie Game Marketing, and An Architectural Approach to Level Design. The GameDev.net FreeBook is relevant to game designers, developers, and those interested in learning more about the challenges in game development. We know game development can be a tough discipline and business, so we picked several chapters from CRC Press titles that we thought would be of interest to you, the GameDev.net audience, in your journey to design, develop, and market your next game. The free ebook is available through CRC Press by clicking here. The Curated Books The Art of Game Design: A Book of Lenses, Second Edition, by Jesse Schell Presents 100+ sets of questions, or different lenses, for viewing a game’s design, encompassing diverse fields such as psychology, architecture, music, film, software engineering, theme park design, mathematics, anthropology, and more. Written by one of the world's top game designers, this book describes the deepest and most fundamental principles of game design, demonstrating how tactics used in board, card, and athletic games also work in video games. It provides practical instruction on creating world-class games that will be played again and again. View it here. A Practical Guide to Indie Game Marketing, by Joel Dreskin Marketing is an essential but too frequently overlooked or minimized component of the release plan for indie games. A Practical Guide to Indie Game Marketing provides you with the tools needed to build visibility and sell your indie games. With special focus on those developers with small budgets and limited staff and resources, this book is packed with tangible recommendations and techniques that you can put to use immediately. As a seasoned professional of the indie game arena, author Joel Dreskin gives you insight into practical, real-world experiences of marketing numerous successful games and also provides stories of the failures. View it here. An Architectural Approach to Level Design This is one of the first books to integrate architectural and spatial design theory with the field of level design. The book presents architectural techniques and theories for level designers to use in their own work. It connects architecture and level design in different ways that address the practical elements of how designers construct space and the experiential elements of how and why humans interact with this space. Throughout the text, readers learn skills for spatial layout, evoking emotion through gamespaces, and creating better levels through architectural theory. View it here. Learn more and download the ebook by clicking here. Did you know? GameDev.net and CRC Press also recently teamed up to bring GDNet+ Members up to a 20% discount on all CRC Press books. Learn more about this and other benefits here.
Sign in to follow this  
Followers 0
Evil Steve

Free Memory Leak Checker (C++)

14 posts in this topic

I've been meaning to tidy this code up for a while and then post it here, but I just never got around to it. Then, today MindWipe was having issues with the CRT memory leak code, so I decided to get of my ass and tidy it up and hand it over. The code was originally in my engine, and I've just removed the engine specific stuff really. So, here is MemMgr.h and MemMgr.cpp. It's a couple of files that you can just drop into any project, rebuild all, and you'll get free memory leak checking code and a few other bits and bobs. You don't need to #include "MemMgr.h" or anything. Example usage:
int main(int, char**)
{
   int* n = new int[42];
   return 0;
}




That'll give you debug spew at application shutdown, and write to memleaks.log, then debug break. The debug spew is the same as you get in memleaks.log, and is (In this case):
1 active allocations totalling 168 bytes:
+ ID 00000002, tag ID 0x00000000: 0x00356B84 168 bytes [main.cpp:9 (main)]
End of allocations




The code only works on Windows, and should work fine on x64. It'll also only work on Visual Studio, due to my #pragma black magic. Only tested on VC2005 so far, but it should work fine in VC2003 or VC2008. A few bits and bobs:
  • It uses a singleton for the memory manager (I know, I'm sorry to all the singleton haters out there), but I believe it's perfectly justified in this case.
  • It uses some #pragma black magic to make sure the memory manager is one of the very first things created, and one of the very last things destroyed. That means it can correctly track STL memory leaks (Although you should only get them if you have e.g. a std::vector in a class allocated with new). This does mean that it might not play nicely in some cases. Please let me know if you find such a case.
  • It does some stack walking with the StackWalk64() function to determine where the allocations came from when memory leaks are detected. It won't give you a full call stack, if you want that it should be straightforwards to modify the code. Instead, it just gives the first function on the stack that isn't in the memory manager, operator new, or STL code.
  • It's thread safe.
  • It only tracks allocations made with new or new[]. It won't handle malloc() allocations (Again, it could be modified to do that I suppose).
  • There's some statistics code you can access if you #include "MemMgr.h" and then access the various functions, e.g. MemMgr::Get().GetAllocatedBytes().
  • The memory manager only compiles into a debug build. Release builds will use the normal operators new and delete. You could change this if you like, but I wouldn't advise it. I'm not sure if this topic should go in Your Announcements, but I figured more people would read it here, and it's a very programming related thing. If any of the mods disagree, feel free to move the topic. If anyone has any questions or problems with it, feel free to post here or PM me. If someone could give it a whirl on VC2003 or VC2008 too, that would be good. EDIT: Updated the code to do runtime linking to RtlCaptureContext; apparently the Platform SDK that comes with VC2008 doesn't know about it. Cheers, Steve [Edited by - Evil Steve on February 14, 2008 8:53:42 AM]
  • 0

    Share this post


    Link to post
    Share on other sites
    Quote:
    It uses a singleton for the memory manager (I know, I'm sorry to all the singleton haters out there), but I belive it's perfectly justified in this case.


    Fortunately for you, it's not a kosher singleton, since it uses explicit and not lazy construction.

    So I guess you're off the hook, and the class is merely a global. Still, at quick glance, you might as well replace the class with namespace, and not lose anything at all.


    Quote:
    It uses some #pragma black magic to make sure the memory manager is one of the very first things created,


    This intrigues me. At least I've never heard of such technique, most other memory managers need to be included at beginning.
    0

    Share this post


    Link to post
    Share on other sites
    Quote:
    Original post by Antheus
    Fortunately for you, it's not a kosher singleton, since it uses explicit and not lazy construction.

    So I guess you're off the hook, and the class is merely a global. Still, at quick glance, you might as well replace the class with namespace, and not lose anything at all.
    That's a good point, I'll add that to my TODO list for it.


    Quote:
    Original post by Antheus
    Quote:
    It uses some #pragma black magic to make sure the memory manager is one of the very first things created,
    This intrigues me. At least I've never heard of such technique, most other memory managers need to be included at beginning.
    Technically, it makes a global in the init_seg section. That global should be the first constructed and the last destructed (If any other globals are in the init_seg the order is undefined). The global creates and destroys the memory manager; the important part is that the memory manager is destroyed after everything else.
    0

    Share this post


    Link to post
    Share on other sites
    I have a very similar memory tracker and is based of C++ for game programmers and it looks like your code is also a derivative of it. I thought that Noel Llopis' code had a license? If not it, it would be nice to just give some credit to the original author.
    0

    Share this post


    Link to post
    Share on other sites
    Quote:
    Original post by dmail
    I have a very similar memory tracker and is based of C++ for game programmers and it looks like your code is also a derivative of it. I thought that Noel Llopis' code had a license? If not it, it would be nice to just give some credit to the original author.
    That's the first I've seen of that actually. In what ways is it similar? A lot of memory leak checkers are based on similar principals.
    0

    Share this post


    Link to post
    Share on other sites
    Quote:
    Original post by Antheus
    Quote:
    It uses a singleton for the memory manager (I know, I'm sorry to all the singleton haters out there), but I belive it's perfectly justified in this case.


    Fortunately for you, it's not a kosher singleton, since it uses explicit and not lazy construction.

    So I guess you're off the hook, and the class is merely a global. Still, at quick glance, you might as well replace the class with namespace, and not lose anything at all.


    What does 'Kosher Singleton' mean, and why is it bad?

    /Curious

    0

    Share this post


    Link to post
    Share on other sites
    Quote:
    Original post by TechnoCore
    What does 'Kosher Singleton' mean, and why is it bad?

    /Curious
    "Kosher" meaning "normal" or "standard" singleton. A normal singleton constructs itself the first time it's referenced, E.g.:

    MemMgr& MemMgr::Get()
    {
    static MemMgr instance;
    return instance;
    }

    Or similarly, with dynamic allocation. In this case, the singleton is created and destroyed explicitly - the key point is that it's not constructed the first time it's used.

    Singletons are usually considered a sign of bad design, since they're really just globals in disguise. See Promit's Journal for some relevant links.
    0

    Share this post


    Link to post
    Share on other sites
    Nice that you release code :)
    Some comments, since I am working on the same thing ATM:
    - __declspec(allocate) may be preferable to #pragma init_seg in that it avoids C4074
    - using an allocation hook instead of overloading new is a bit more robust (what happens if you link against another library that decides to do the same?) and also covers use of malloc/getcwd/strdup etc.

    More importantly: we certainly can't hope to reach Valgrind's degree of studliness, so it makes sense to use that on Linux and otherwise consider the MS and Mac platform libraries. The MS debug heap already does all of this and more - except for recording a call stack. Wouldn't it be nifty to reuse that code, just modifying it to also gather a stack trace? This is what the earlier, safer versions of VLD did.

    In fact, we can take this approach one step further. Storing 64 bytes of caller information *for each allocation* is a bit hard to justify, especially if there are lots of allocations. (what good is debug mode if you can't run it or it's just too slow?) It turns out that several frames can be stored with *zero* space overhead: we just need to stash them in the memory block header's file and line fields. Those 64 bits (for 32-bit builds) are enough to store at least two full 32-bit pointers* or up to six encoded offsets, the first being relative to the code segment.

    * dirty trick: you need one bit to differentiate the pointer vs offset, but get it back by not storing the LSB (since no call instruction can be less than two bytes, rounding the return address down to the next even address does not change the function).


    > today MindWipe was having issues with the CRT memory leak code
    Out of curiosity, what kind of problems?
    0

    Share this post


    Link to post
    Share on other sites
    Quote:
    Original post by Jan Wassenberg
    Nice that you release code :)
    Some comments, since I am working on the same thing ATM:
    - __declspec(allocate) may be preferable to #pragma init_seg in that it avoids C4074
    - using an allocation hook instead of overloading new is a bit more robust (what happens if you link against another library that decides to do the same?) and also covers use of malloc/getcwd/strdup etc.

    More importantly: we certainly can't hope to reach Valgrind's degree of studliness, so it makes sense to use that on Linux and otherwise consider the MS and Mac platform libraries. The MS debug heap already does all of this and more - except for recording a call stack. Wouldn't it be nifty to reuse that code, just modifying it to also gather a stack trace? This is what the earlier, safer versions of VLD did.

    In fact, we can take this approach one step further. Storing 64 bytes of caller information *for each allocation* is a bit hard to justify, especially if there are lots of allocations. (what good is debug mode if you can't run it or it's just too slow?) It turns out that several frames can be stored with *zero* space overhead: we just need to stash them in the memory block header's file and line fields. Those 64 bits (for 32-bit builds) are enough to store at least two full 32-bit pointers* or up to six encoded offsets, the first being relative to the code segment.

    * dirty trick: you need one bit to differentiate the pointer vs offset, but get it back by not storing the LSB (since no call instruction can be less than two bytes, rounding the return address down to the next even address does not change the function).


    > today MindWipe was having issues with the CRT memory leak code
    Out of curiosity, what kind of problems?
    Interesting, I didn't know about allocation hooks or __declspec(allocate), I'll have a look at them. I agree that storing 64 bytes (Actually, it's more with the stack trace contexts [rolleyes]) is a bit much for each allocation, but I haven't run into any problems with it (yet).

    The problem MindWipe was having turned out to be a genuine memory leak (GLUT never returning, so memory never being deleted), but I thought it was because some global was getting destroyed after the _CrtDumpLeakCheck (Or whatever the function is).
    0

    Share this post


    Link to post
    Share on other sites
    Another thing you can do is store the stack traces separately. This is so that if you have a lot of memory being allocated at a particular point in code with a unique "call stack signature", you don't have to store it on a per-allocation basis, but rather just once in the unique list and have the allocation point to its stack trace. Our engine at work uses a std::hash_map (with a custom allocator of course) to store the stack traces with their CRC as a key. Recursion guards are used to prevent memory allocations from within the memory manager from causing infinite loops, while also allowing us to track the amount of memory in use by the memory manager itself. This isn't something to overlook -- at one point we thought our engine was leaking 28 megabytes, until we discovered all that memory was in use by the leak tracking system :)
    0

    Share this post


    Link to post
    Share on other sites
    Quote:
    This intrigues me. At least I've never heard of such technique, most other memory managers need to be included at beginning.

    The ones that require including at the beginning will typically use macros and redefine new/malloc, which breaks placement new and member functions that happen to be called free, etc.
    The init_seg / __declspec(allocate) technique is useful for performing init before main, before constructors run, or even before the CRT modules are initialized. See http://blogs.msdn.com/larryosterman/archive/2004/09/27/234840.aspx for an explanation.

    Incidentally, Steve overrides global operator new, which happens at link-time and requires no init, but the synchronization object and dbghelp do. BTW, another advantage of allocation hooks is that they are called while under the CRT _HEAP_LOCK, which goes a long way towards making a memory manager thread-safe without any further work.

    Quote:
    The problem MindWipe was having turned out to be a genuine memory leak (GLUT never returning, so memory never being deleted), but I thought it was because some global was getting destroyed after the _CrtDumpLeakCheck (Or whatever the function is).

    Ah, ok. You can pretty much rely on the CRT report because it has the final word on shutdown order.

    Zipster: good idea. 28 MB in spite of this optimization, though? That is an excellent motivation for me to finish up the zero storage scheme :)
    0

    Share this post


    Link to post
    Share on other sites
    Whoa, got some freaky deja vu. Probably from Jan Wassenberg's old post when he announced his manager and discussed some of the same issues.

    Because Evil Steve, you haven't mentioned this before?

    [Edited by - Boder on February 19, 2008 1:21:49 PM]
    0

    Share this post


    Link to post
    Share on other sites
    Quote:
    Original post by Boder
    Because Evil Steve, you haven't mentioned this before?
    I've posted a similar memory manager before, but it was just straight from my engine code and needed tweaking to get it to compile; it was intended as a code sample for something or other, not something people can download and drop into their code like this.
    0

    Share this post


    Link to post
    Share on other sites
    Quote:
    Original post by Jan Wassenberg
    Zipster: good idea. 28 MB in spite of this optimization, though? That is an excellent motivation for me to finish up the zero storage scheme :)

    Yes, however I misspoke. It was the allocations we were storing in a hash map, and the stack traces were in a static array. This array was about 21 MB, and the hash map overhead was another 7 MB or so. However we did manage to fill the entire stack trace array on a few occasions, so ~28 MB was indeed in active use at those times.

    I should mention that this was the usage we measured using the release CRT. The amount of additional information we stored with each allocation was much less than a debug CRT header, so there really wasn't much more voodoo magic we could do besides shortening the call stack depth or trying a different container. Believe me we could have used a few tricks, because we were running up against hard memory limits on our target platform :)
    0

    Share this post


    Link to post
    Share on other sites
    Quote:
    Original post by Evil Steve
    "Kosher" meaning "normal" or "standard" singleton. A normal singleton constructs itself the first time it's referenced, E.g.:
    *** Source Snippet Removed ***Or similarly, with dynamic allocation. In this case, the singleton is created and destroyed explicitly - the key point is that it's not constructed the first time it's used.

    Singletons are usually considered a sign of bad design, since they're really just globals in disguise. See Promit's Journal for some relevant links.


    Thanks for the clarification... you had me reading for a couple of hours there ;)

    0

    Share this post


    Link to post
    Share on other sites

    Create an account or sign in to comment

    You need to be a member in order to leave a comment

    Create an account

    Sign up for a new account in our community. It's easy!


    Register a new account

    Sign in

    Already have an account? Sign in here.


    Sign In Now
    Sign in to follow this  
    Followers 0