Sign in to follow this  

tracking memory leaks

This topic is 4749 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hello. Up till now iv been using _CrtMemState to track memory leaks, one at the beginning of main() and one at the end. This approach feels pretty limited, i generaly check the output once in a while and if it has pickedup a leak i sit and comment large portions of code out until i can pin-point the leak. What im wondering is if anyone is familiar with a better way, iv heard there are 3rd party external apps that you can hookup to the process or debugger to track memory. Are there any free/good ones? I know there is also the possibility to overload new&delete and build your own custom memory tracker but i would rather not have to do that if possible. Or maybe there is some other, better way? Thanks in advance.

Share this post


Link to post
Share on other sites
I don't know of any free external application, but I suggest taking a look at this. It is a free memory manager by Paul Nettle that you can integrate to existing applications quite easily.

Share this post


Link to post
Share on other sites
[edit] Aldenar's typing skills > mine [embarrass][/edit]
What I would do is use a custom memory leak tracking library. I am trying out Paul Nettle's, I have heard good things about it. It was here on flip code but the link no longer works. I did a google search and was unable to find any links that worked, so I have uploaded it to my web page. To use it, all you have to do is:

#include "mmgr.h"

then at the end of your program:

m_dumpMemoryReport();

I would *strongly* suggest reading the source code files b/c they explain some important stuff. I hope this helps!

Share this post


Link to post
Share on other sites
Official link here Fluid Studios MMGR. Apparently Fluid Studios took Nettle's code and updated(?) it?

It integrates VERY nicely in my own engine, and it has even detected memory leaks in some libraries I use- I recompiled the libraries with the MMGR to pinpoint the leak (on VC7 it tells you file, function, and line) and then fixed them. Read the docs in the source. Make sure #include "mmgr.h" is the very last include in each source file, and don't include it in the headers: STL doesn't like being memory managed :) This is contrary to the docs, but after butting my head against the official instructions I realized that this accomplishes the same thing without any hassle.

MMGR dumps a file telling you your leaks at the end of execution, as well as a file logging each allocation and deallocation if STRESS_TEST is enabled. These files are complete with file, function, and line, of course. It is probably easiest to include it in your project, but you can compile it as a library if you don't need to edit the flags regularly.

Quote:
then at the end of your program:
m_dumpMemoryReport();

Not in this version, at least... all you do is include mmgr.h and it logs leaks automagically

Share this post


Link to post
Share on other sites
For anyone using the CRT tools (I've had to use them when MMGR does not co-operatre with some library):

When you get a leak (or other heap error), you'll get some number in squiggly brakcets like {2342}. First run your program a few times more to check it's the same number every time. What you do is take that number, and at the top of your main function, drop in:

_CrtHaltOnAloc(2342);

Which will then assert just before that particular peice of memory is alocated. That will tell you what object is being leaked, and will help you a great deal in fixing it.

Note: That function name is from memory - it could be something similar.

MMGR is much better, it's unfortunatly less compatable than the CRT stuff, though.

Share this post


Link to post
Share on other sites
Not that I want to hijack a thread, but I use the MMGR header/code... absolutely genius bit of work. However, I get a random number of leaks reported each time I run the program - often only 5 or 6. None of them can be identified in the log (the source is ????::????)

However, MMGR is configured fine for every translation unit and has picked up every other leak (my program was a bit of a sieve before I used it!). Anyone ever seen it do this?

I keep wondering if it's picking up something that initialized externally and managed by something else (e.g. the STL), but I haven't managed to find a good answer yet.

cheers,
Jack

Share this post


Link to post
Share on other sites
 ----------------------------------------------------------------------------------------------------------------------------------
| Memory leak report for: 12/16/2004 12:10:42 |
----------------------------------------------------------------------------------------------------------------------------------


28 memory leaks found:

Alloc. Addr Size Addr Size BreakOn BreakOn
Number Reported Reported Actual Actual Unused Method Dealloc Realloc Allocated by
------ ---------- ---------- ---------- ---------- ---------- -------- ------- ------- ---------------------------------------------------
000196 0x089303F0 0x0000001C 0x0892F3F0 0x0000201C 0x00000000 new N N cubedemo.cpp(00186)::CubeDemo::init
000093 0x089406E0 0x00000028 0x0893F6E0 0x00002028 0x00000000 new N N ??(00000)::??
...

The discrepancy between the reported size and actual size comes from MMGR adding an extra 0x2000 bits (1024 byte)of memory to each allocation for testing: you can change this behavior with a flag in mmgr.cpp.

You get the unknown location of the leak when the file the leak is in is not including "mmgr.h" - ie, in an external library you are linking with. If you can deduce what library the leak is in, recompile it with "mmgr.h" included in all source files and relink your app. It will now use the #defines in mmgr.h to pinpoint the leak.

I haven't had any trouble with the STL leaking in the past... at the moment, as my log shows, I have 28 leaks in some library or another (I just went on a functionality spree and added about 6 libraries, debugging this will be fun). I guess it is technically possible that it's STL, but it is impossible to debug it (STL barfs if you try to memory manage it - try including "mmgr.h" before any of your STL includes and you shall see).

Share this post


Link to post
Share on other sites
Quote:
You get the unknown location of the leak when the file the leak is in is not including "mmgr.h" - ie, in an external library you are linking with. If you can deduce what library the leak is in, recompile it with "mmgr.h" included in all source files and relink your app. It will now use the #defines in mmgr.h to pinpoint the leak.
Yup, I read the files on how it works, but this is my point - my entire project has mmgr.h included correctly (it states to #include after ALL other #include's).

I still get the leaks appearing with no identity:

343 memory leaks found:

Alloc. Addr Size Addr Size BreakOn BreakOn
Number Reported Reported Actual Actual Unused Method Dealloc Realloc Allocated by
------ ---------- ---------- ---------- ---------- ---------- -------- ------- ------- ---------------------------------------------------
001894 0x011F0708 0x00000024 0x011F06F8 0x00000044 0x0000001C new N N ??(00000)::??
001898 0x011F0788 0x00000024 0x011F0778 0x00000044 0x00000008 new N N ??(00000)::??
000992 0x011E0798 0x00000024 0x011E0788 0x00000044 0x0000001C new N N ??(00000)::??
000993 0x011E0818 0x0000000C 0x011E0808 0x0000002C 0x00000004 new N N ??(00000)::??


The only three libraries that my program currently uses are DirectX 9.0, STL and Win32 API.

Given that unless mmgr.h is missing something in a file that it's #include'd in, it's got to be coming from somewhere thats NOT my code.

Also, I'm pretty damn sure it's the same thing that keeps leaking as the sizes are almost always 0x44 (68 bytes) or 0x2C (44 bytes)...

Any thoughts on what might cause mmgr.h to detect a leak but not identify it?

Cheers,
Jack

Share this post


Link to post
Share on other sites
Hi

Very helpful thread so far... I've tried previously mentioned Paul Nettle's memory manager and it indeed found 3 stupid leaks in my code :-) however, it also reports some wierd anonymous leaks (just like Jollyjeffers got):



7 memory leaks found:

Alloc. Addr Size Addr Size BreakOn BreakOn
Number Reported Reported Actual Actual Unused Method Dealloc Realloc Allocated by
------ ---------- ---------- ---------- ---------- ---------- -------- ------- ------- ---------------------------------------------------
000037 0x04C99800 0x00000D68 0x04C997F0 0x00000D88 0x00000B80 new N N ??(00000)::??
000039 0x04C9A5E0 0x00001480 0x04C9A5D0 0x000014A0 0x0000109C new N N ??(00000)::??
000002 0x003FB540 0x00000280 0x003FB530 0x000002A0 0x0000008C new N N ??(00000)::??
000003 0x003FB7E8 0x000003E8 0x003FB7D8 0x00000408 0x00000028 new N N ??(00000)::??
000052 0x04C9CB20 0x00001708 0x04C9CB10 0x00001728 0x000010BC new N N ??(00000)::??
000026 0x003FDC70 0x000007E8 0x003FDC60 0x00000808 0x0000058C new N N ??(00000)::??
000114 0x04C9F880 0x000006F8 0x04C9F870 0x00000718 0x0000026C new N N ??(00000)::??




Tested application uses only my own engine (which proofed to be leak-safe, as none of those leaks link in any way with its files) which in turn uses SDL, OpenGL, SDL_TTF, FMOD, TinyXML and SDL_Image. It may be (but I don't believe it) that one of theese libs produces leaks... hmmm, there's also a chance that it was caused by compiler, which used some black magic - hacking to achieve its goals, and allocations were cleaned, but ie. after program was about to exit (but not normal main() exit, rather then before immediate return to OS).

btw, thx Thedustbustr for:

Quote:
Make sure #include "mmgr.h" is the very last include in each source file, and don't include it in the headers: STL doesn't like being memory managed :) This is contrary to the docs, but after butting my head against the official instructions I realized that this accomplishes the same thing without any hassle


which saved my butt: first I read the docs and typed #include "mmgr.h" before everything else in all my engine's files (and there are many of them, SC now has more than 450kb...) but it didn't compile - got strange errors during compilation. Luckily, I saw your post... so I tried for second time shifting all those includes (that was truly boring...) down - and it compiled without any warning ;-)

Share this post


Link to post
Share on other sites
If you define STRESS_TEST in mmgr.cpp it dumps every allocation and deallocation in order- maybe you can infer from the allocation number approximately where in your code the leak is? Of course, it's probable that you will see all your constructors then get all your ?? and then see all your destructors, which isn't very helpful :(

A possibility is that you are misusing a directx function: I've never used DX, but there might be a function meant to be paired with its 'antifunction', where one news and the other deletes, but you aren't calling the delete. Something along the lines of
while (!exit) D3Dvector v = NewD3DVectorPointer();


Try what Andrew Russel said:
Quote:
For anyone using the CRT tools (I've had to use them when MMGR does not co-operatre with some library):

When you get a leak (or other heap error), you'll get some number in squiggly brakcets like {2342}. First run your program a few times more to check it's the same number every time. What you do is take that number, and at the top of your main function, drop in:

_CrtHaltOnAloc(2342);



Koshmaar said,
Quote:
It may be (but I don't believe it) that one of theese libs produces leaks..
Why don't you believe it? Memory leaks probably aren't high on their priority leak to fix, if it isn't leaking every frame: if its a matter of calling delete ptr; before the program returns, its effectively pointless because the memory is freed anyway when the program exits. Be worried if the number of leaks is proportional to runtime. The GLText lib that comes with Game Programming Gems 1 had such a leak in it: i squashed it because it annoyed me to have MMGR report 1 leak :)

Share this post


Link to post
Share on other sites
Quote:
If you define STRESS_TEST in mmgr.cpp it dumps every allocation and deallocation in order- maybe you can infer from the allocation number approximately where in your code the leak is? Of course, it's probable that you will see all your constructors then get all your ?? and then see all your destructors, which isn't very helpful
I'll have a look at this one, wasn't aware of the #define option existing!

Quote:
Try what Andrew Russel said:

Quote:

For anyone using the CRT tools (I've had to use them when MMGR does not co-operatre with some library):

When you get a leak (or other heap error), you'll get some number in squiggly brakcets like {2342}. First run your program a few times more to check it's the same number every time. What you do is take that number, and at the top of your main function, drop in:

_CrtHaltOnAloc(2342);


I've been meaning to try this one for ages now [wink], keep forgetting though. Hopefully it'll find what I'm after.


What I was theorizing for a long time was that, as hinted in the docs somewhere, that statically/const allocated memory (which is created right at the start of execution usually) is also only removed right at the end of the program; i.e. before and after the memory manager works its magic. If this is the case, it might just be an erronous report - and the leaks don't actually exist.

Yet, if I read the code correctly, mmgr.cpp should only be able to tag/log/output a given memory allocation IF it's "new" operation is overloaded accordingly. And in order for that to be the case, mmgr.h should be included at the some point in that translation unit. Confuses me with circular/contradictory logic at this point [grin].

Cheers for the help thedustbustr - appreciated!
Jack

Share this post


Link to post
Share on other sites
Quote:
Yet, if I read the code correctly, mmgr.cpp should only be able to tag/log/output a given memory allocation IF it's "new" operation is overloaded accordingly. And in order for that to be the case, mmgr.h should be included at the some point in that translation unit. Confuses me with circular/contradictory logic at this point.

My own understanding is thus, use it at your own risk. I probably have no idea what I am spewing :)

mmgr.h solely redefines 'new' to their own version of '*operator new' which takes three arguments (__FILE__,__FUNCTION__,and __LINE__). These arguments are defines so the compiler passes them in automatically whenever you call just plane 'new'. However, when you link a file without these defines, the new is called without arguments because it hasnt been redefined to take arguments. However, all your apps will used the linked new instead of the msvcrt new. Every allocation goes through MMGR, but the only ones that are trackable are those with mmgr.h included.

According to this rationale, EVERY memory allocation in the app is logged by MMGR, in every piece of code executed. For it to be properly tagged/tracked, it must use the new with arguments, but new without arguments will still register.

This makes sense to me, so maybe there is truth to it :)

jollyjeffers, I noticed this while reading the code again:
// ---------------------------------------------------------------------------------------------------------------------------------
// -DOC- When tracking down a difficult bug, use this routine to force a breakpoint on a specific allocation count
// ---------------------------------------------------------------------------------------------------------------------------------

void m_breakOnAllocation(unsigned int count)
{
breakOnAllocationCount = count;
}
You will probably find that useful for tracking your bug... without looking into it, I believe that the allocation count is the alloc number in the log. It seems to accomplish the same thing as that crt routine. I must admit that I couldn't get it to work tonight, I might have another go tomorrow after I get some sleep. It isn't defined in the header... I tried defining it but it had no apparent effect on execution.

Another useful thing I noticed tonight is
m_dumpMemoryReport("memreport.log",true);
Call this as the very last thing before your program returns. Dumps very nice statistics not displayed by default.

Share this post


Link to post
Share on other sites
Quote:
EVERY memory allocation in the app is logged by MMGR, in every piece of code executed. For it to be properly tagged/tracked, it must use the new with arguments, but new without arguments will still register.
Interesting, my cursory exam of the mmgr.cpp source code led me to believe it only adds allocations (That are eventually outputted to the log) that it overloads. If it catches everything then that could explain what we're seeing.


Quote:
ou will probably find that useful for tracking your bug... without looking into it, I believe that the allocation count is the alloc number in the log. It seems to accomplish the same thing as that crt routine.
I'll have a mess with this one as well then.


Quote:
Another useful thing I noticed tonight is

m_dumpMemoryReport("memreport.log",true);

Call this as the very last thing before your program returns. Dumps very nice statistics not displayed by default.


I'm going to have to claim that its been (by my change log) around 7 months since I implemented mmgr into our project, hopefully I've just forgotten that all these things existed - and wasn't stupid enough to miss them completely [embarrass]. Thanks for pointing them all out though!

Cheers,
Jack

Share this post


Link to post
Share on other sites
Oh, it catches everything. Look at one of the header files and you'll notice that it actually defines a 'new' macro, so anyplace in your code (or code that your code calls) that calls 'new' ends up calling that system wide macro, which then allocates the memory in MMGR. It's a bit of a hack, but it works.


- Houdini

Share this post


Link to post
Share on other sites

This topic is 4749 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this