Sign in to follow this  
Samith

Tracking memory leaks in C++

Recommended Posts

Samith    2460
Memory leaks as we all know are an annoying problem. But again I'm sort of hung up trying to think of a way to avoid them. I know how smart/intrusive pointers work and how to use them, but this feels like an incomplete solution. If I forget to use a smart pointer I could potentially leave some memory undeleted, thus causing a leak. I'm wondering what people have used as solutions to memory problems. Ideally I'd like some sort of solution that causes a very clear runtime error if a memory leak is allowed. It should allow me to see what memory I forgot to delete, etc. Currently the only solution I can think of is to allocate some big pool of memory at the beginning of my program, then force (somehow, overloaded new perhaps?) everyone to use that pool and tag the memory they allocate so that at the end of the program I can see if there are any remaining tags. The problem is that this solution seems somewhat inflexible. Is this much security even necessary? Are there other possible solutions that anyone knows of? I know this is a very broad subject and the correct solution depends highly on what type of application is being written, but I think for now I'd just like to see what solutions people have used to give me an idea of what's out there. Any ideas would be appreciated. Sam

Share this post


Link to post
Share on other sites
SiCrane    11839
Your compiler/runtime library may have leak tracking functionality built in. For example with MSVC you can use _CrtDumpMemoryLeaks() or _CrtSetDbgFlag ( _CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF ).

Share this post


Link to post
Share on other sites
Samith    2460
Wow, thanks for the fast responses. Those look like very handy tools. Are these Microsoft specific, though? I am currently using MSVC but I would also like to know some solutions that would be portable to other development environments.

Great advice, though, I will certainly try to make use of this functionality.

Share this post


Link to post
Share on other sites
Verg    450
Paul Nettle's Memory Manager has worked well for me.

As far as the smart pointer discussion goes... why not just use all smart pointers?

If you have a lot of refactoring to do to get to that point... you can start to introduce them a little at a time until you get your memory leaks under control.

The Loki Library is lightweight and has a very nice Smart pointer template. boost::shared_ptr is a great option too.

Share this post


Link to post
Share on other sites
UltimateWalrus    229
This is a pretty simple solution. But it works for me a lot of the time.

Create two functions:


int Allocations = 0;

inline void *dmalloc(unsigned long size)
{
Allocations++;
return malloc(size);
}

inline void dfree(void *pointer)
{
Allocations--;
free(pointer);
}


Or something like that. If you want to use "new" and "delete" that's OK too. I haven't compiled this, but it should work.

At the end of the program, just check the "Allocations" variable; it should be 0. If it's not, you have a memory leak (although you'll have to figure out where by yourself). ;)

Share this post


Link to post
Share on other sites
fpsgamer    856
Quote:
Original post by UltimateWalrus
At the end of the program, just check the "Allocations" variable; it should be 0. If it's not, you have a memory leak (although you'll have to figure out where by yourself). ;)


That method is somewhat problematic. Consider the fact that you may have pointers that last for the entire life of the application. You do not have to bother deleting these since all memory your application claims during execution is returned to the system after termination anyways.

A memory leak involves you "losing handles" to memory during execution

ie.

int* a = new int[100];
a = (int*)5; // how the hell do we access our array now?

Now that memory is now lost for the entire execution of the application.

Using a more advanced tool will give you information about which pointers have not been free'd, not just that some stuff hasn't been free'd.

Share this post


Link to post
Share on other sites
Evil Steve    2017
My memory manager; Overloads operators new and delete, has some stack walking functions to determine where allocations came from, is thread safe, and works with X64:
PMemory.h, PMemory.cpp.
That code won't compile straight out of the box, but it should with some minor modifications (E.g. removing my assert calls and so on). It should work with other compilers (untested), the only MSVC specific part I know of is the #pragma init_seg(compiler) call (Which is there to ensure the memory manager global is constructed before STL globals, so it can track STL "leaks").

Share this post


Link to post
Share on other sites
fpsgamer    856
Quote:
Original post by Evil Steve
My memory manager; Overloads operators new and delete, has some stack walking functions to determine where allocations came from, is thread safe, and works with X64:
PMemory.h, PMemory.cpp.
That code won't compile straight out of the box, but it should with some minor modifications (E.g. removing my assert calls and so on). It should work with other compilers (untested), the only MSVC specific part I know of is the #pragma init_seg(compiler) call (Which is there to ensure the memory manager global is constructed before STL globals, so it can track STL "leaks").


Have you considered releasing the EvilSteve standard library so we can all easily accumulate all the code you've posted? [grin]

Share this post


Link to post
Share on other sites
UltimateWalrus    229
Quote:
Original post by fpsgamer
That method is somewhat problematic. Consider the fact that you may have pointers that last for the entire life of the application. You do not have to bother deleting these since all memory your application claims during execution is returned to the system after termination anyways.

A memory leak involves you "losing handles" to memory during execution

ie.

int* a = new int[100];
a = (int*)5; // how the hell do we access our array now?

Now that memory is now lost for the entire execution of the application.

Using a more advanced tool will give you information about which pointers have not been free'd, not just that some stuff hasn't been free'd.


But isn't it still good practice to clean up all your memory resources when the program terminates? I'm just a novice, so you probably know more about it than I do :)

I don't doubt that an advanced tool would be useful (especially for a large-scale game). But rather than downloading a whole framework for my little projects, I'd rather just try to use good programming practice. Every time I make an allocation, I usually go down right away and put a free somewhere. Granted, this won't prevent the error you described (setting the pointer to a new value), but it can prevent memory leaks, say, when a pointer variable goes out of scope. My point is, there are ways of programming that can minimize the risks of memory leaks.

Share this post


Link to post
Share on other sites
Evil Steve    2017
Quote:
Original post by UltimateWalrus
But isn't it still good practice to clean up all your memory resources when the program terminates? I'm just a novice, so you probably know more about it than I do :)
Yup, definitely.

Quote:
Original post by UltimateWalrus
My point is, there are ways of programming that can minimize the risks of memory leaks.
Minimise, yes. Remove, no. This is just one cause of a memory leak, there's loads of other things that can cause them.

Share this post


Link to post
Share on other sites
fpsgamer    856
Quote:
Original post by UltimateWalrus
Quote:
Original post by fpsgamer
That method is somewhat problematic. Consider the fact that you may have pointers that last for the entire life of the application. You do not have to bother deleting these since all memory your application claims during execution is returned to the system after termination anyways.

A memory leak involves you "losing handles" to memory during execution

ie.

int* a = new int[100];
a = (int*)5; // how the hell do we access our array now?

Now that memory is now lost for the entire execution of the application.

Using a more advanced tool will give you information about which pointers have not been free'd, not just that some stuff hasn't been free'd.


But isn't it still good practice to clean up all your memory resources when the program terminates? I'm just a novice, so you probably know more about it than I do :)

I don't doubt that an advanced tool would be useful (especially for a large-scale game). But rather than downloading a whole framework for my little projects, I'd rather just try to use good programming practice. Every time I make an allocation, I usually go down right away and put a free somewhere. Granted, this won't prevent the error you described (setting the pointer to a new value), but it can prevent memory leaks, say, when a pointer variable goes out of scope. My point is, there are ways of programming that can minimize the risks of memory leaks.


Ya, I think my phrasing was a little strong.

That method isn't as bad an I may have made it sound.

Share this post


Link to post
Share on other sites
Verg    450
Quote:
Original post by UltimateWalrus
I don't doubt that an advanced tool would be useful (especially for a large-scale game). But rather than downloading a whole framework for my little projects,


Paul Nettle's is easy and consists of two (well, three) files. Evil Steve's (obviously) consists of just two files. It's worth it [smile]

Share this post


Link to post
Share on other sites
Mike2343    1202
Quote:
Original post by Evil Steve
My memory manager; Overloads operators new and delete, has some stack walking functions to determine where allocations came from, is thread safe, and works with X64:
PMemory.h, PMemory.cpp.
That code won't compile straight out of the box, but it should with some minor modifications (E.g. removing my assert calls and so on). It should work with other compilers (untested), the only MSVC specific part I know of is the #pragma init_seg(compiler) call (Which is there to ensure the memory manager global is constructed before STL globals, so it can track STL "leaks").


You could wrap that #pragma in a #ifdef or simular to make it only included in MSVC compilers? I haven't looked at your code but might be an idea. Just for your sake later too?

Share this post


Link to post
Share on other sites
taby    1265
If you're absolutely worried about memory leaks, use vector's contiguous memory layout. Deconstructor takes care of memory release, removing the need for pointer-based arrays.

When allocating/instantiating a single object through a pointer though, the extra memory and speed requirement of the vector class may not be worth it.

Also, one could build a registration class which tracks memory allocation through pointers by exposing member functions that are used in proxy of direct calls to new/new[] and delete/delete[]. This way the member function can examine the existing state of the pointer in question, and deallocate it if need be before allocating.


#include <iostream>
using std::cout;
using std::endl;

#include <new>
using std::nothrow;

template<class T>
bool array_alloc(T *&ptr, size_t elems = 0)
{
// T **ptr_to is used to compare against the database.
// Memory cost of tracking this pointer in the database
// is sizeof(T**).
T **ptr_to = &ptr;

bool is_allocated = false;

if(0 == elems)
return is_allocated; // false

// If this address is found in the database,
// it is passed to delete[].
if(0 != *ptr_to)
delete [] *ptr_to;

// Upon failure, return 0 instead of throwing std::bad_alloc.
*ptr_to = new(nothrow) T[elems*sizeof(T)];

if(0 == *ptr_to)
return is_allocated; // false;
else
is_allocated = true;

for(size_t i = 0; i < elems; i++)
(*ptr_to)[i] = 0;

return is_allocated; // true
}

int main(void)
{
size_t size = 10;

char *y = 0;

array_alloc(y, size);

cout << static_cast<int>(y[size - 1]) << endl;

// When the database class goes out of scope,
// all known allocated pointers are passed to delete[].

return 0;
}





Edit: changed the source.

[Edited by - taby on August 5, 2007 11:57:23 PM]

Share this post


Link to post
Share on other sites
David Neubelt    866
Quote:
Original post by Zahlman
Or, just don't create memory leaks. :)


I wish it was that simple life would be great :P

I just spent the last three weeks writing custom software to track memory leaks and you'd be suprised on how many different ways you can leak memory.

The solution I've come up with is hook into new/placement new/malloc and your OS's allocation calls. Then write a function to walk up the stack, and based on the program counter look up the function name in the map file for the executable. This has given me the quickest way to track down memory leaks.

-= Dave

Share this post


Link to post
Share on other sites
Deaken Frost    133
If you are using MSVC you can also use Visual Leak Detector (http://dmoulding.googlepages.com/vld). It's very easy to use and the least intrusive method I have seen. It only works in debug mode, however.

Share this post


Link to post
Share on other sites
moosedude    145
If you're having trouble with memory, maybe consider a different design methodology for your new programs that naturally enforce memory construction/destruction rules. Each time I create a new class I write a standard set of functions that are called on construction and destruction of the class, it works very well.

Failing that, if you're considering overload the memory allocation functions, you may as well go the whole journey and create your own simple memory manager, that way you can be exceptional messy when allocating memory and just delete your memory manager pool at the end of the program. I personal think this is a brillant way of creating prototypes really quickly when you don't want to think about memory management.

Share this post


Link to post
Share on other sites
ViperG    206
I'm actually switching all my pointers back to structs/class/vars etc and pushing them into STL's, plus im removing all my char* and changing them to stl string.!!!

That should take care of my memory leaks if I had any anyways.

no longer need NEW and Delete and Malloc SWEET!

Share this post


Link to post
Share on other sites
David Neubelt    866
Quote:
Original post by ViperG
I'm actually switching all my pointers back to structs/class/vars etc and pushing them into STL's, plus im removing all my char* and changing them to stl string.!!!

That should take care of my memory leaks if I had any anyways.


I've heard this a few times already in this post. It's not true. Simply using STL, RTAII, and smart pointers wont resolve all memory leaks. I'd like to also point out restricting your self to only those three allocation patterns limits your design choices.

There are other ways you can get into trouble with memory leaks so it's still important to have memory tracking solutions. One example, say you're using polymorphic behavior in an stl container object. You store a bunch of base smart pointers to dynamically created derived instances. Normally, you'd expect that when the destructor of the class containing the vector is destroyed it would clean up all the memory of the smart pointers. Simply forgetting a virtual destructor in the base class can cause a memory leak.

Another class of memory leaks is from something more tricky that STL, RTAII, and smart pointers wont pick up. I've call them logic leaks. For example, imagine you are playing a game that is level based. You'd expect that after each level anything contained in the level is cleaned up. If everything that was referenced in the level is contained within a level object then you can trust that deleting the level will clean everything up. The problem comes from allocations that have to be made outside of the level for events, and logic that goes across multiple levels and can not be referenced from the level. These types of memory leaks may get cleaned up when the game exits but as you're playing will grow more and more until eventually you run out of memory. These, in my opinion, are the hardest to track.

Also, there is a class of memory leaks within a level from soft resets. I define a soft reset, as a player death. Where you don't want to reload the entire level but just restart the state of all the logic and events since the last level check point. If you simply destroyed and reloaded the level each time you died then you'd cause load times. Not correctly handling which items that need to be flushed can cause memory to increase if the player dies a lot in one area.

There are others as well but my point is that investing into either writing your own memory tracking solution or using an existing one is a good investment if you want to deliver solid software.

Share this post


Link to post
Share on other sites
Zahlman    1682
Quote:
Original post by David Neubelt
Quote:
Original post by Zahlman
Or, just don't create memory leaks. :)


I wish it was that simple life would be great :P

I just spent the last three weeks writing custom software to track memory leaks and you'd be suprised on how many different ways you can leak memory.


The problem is that you're writing code thinking about how to avoid all the many different ways to leak memory, instead of writing code thinking about how to make use of the few very nice ways to *not* leak memory. Smart pointers and standard library containers are your friends. Not to mention simplifying your design, by not adding useless levels of indirection.

Share this post


Link to post
Share on other sites
Antheus    2409
Quote:
These types of memory leaks may get cleaned up when the game exits but as you're playing will grow more and more until eventually you run out of memory. These, in my opinion, are the hardest to track.


There are design solutions to that - unfortunately they are bulky and will impose functionality, performance and design compromises.

If someone is still coding in C++ for performance, the only performance benefits come from abandoning safe concepts to do it "the fast way". After all, Java's JVM is written in C or C++. So various degrees of safety are available in C++.

But good design practices will go a long way.

Ironically enough, garbage collection (system-wide, not just smart pointers) is very suited for solving the longevity problems. But the problem of exhausting global heap pool remains. And hybrid systems would likely cause more problems than they solve (although C# seems to cover that pretty well, there's still plenty of gotchas once you dig deep enough).

But still - spending time debugging logic errors will likely be more productive than chasing down an obscure dangling pointer that is managed by 17 different classes.

Share this post


Link to post
Share on other sites
speciesUnknown    527
I love low tech solutions to these kind of problems, and heres mine. Its not the most elegant solution but it works.

Supposing your application has several stages in the game loop:

read input devices
handle physics
handle AI
update game
render objects
update screen

And you have a gradual increase in memory usage, but you don't know where.

Cut that list in half and do two separate loops with half those steps:

//first half
read input devices
handle physics
handle AI

//second half
update game
render objects
update screen

you now know which half leaks memory faster, and which half, if any, does not leak memory.
Once you pinpoint a single step, or several individual steps, do the same within those steps. Narrow it down to a few stages and fix the problems.

[edited] a couple of typos

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this