Avoiding heap fragmentation?

Started by
5 comments, last by Red Ant 15 years, 10 months ago
At work, when I have the opportunity, I have been replacing older C-style code with C++ code. I work on a high availability application and I am instructed, where possible, to perform all dynamic memory allocation during initialization. Aside from minimizing the possibility of a memory leak (which is moot when dealing with standard contaniers) the concern is for heap fragmentation that multiple dynamic allocations/deallocations can cause over the days, weeks and months that the application continuously runs. In many cases this is not a problem as I use a container's reserve member to pre-allocate the amount of space the old buffers were using at start time. My issue, however, is what to do when I need to declare a container within a method? I can do something ugly like making class members out of buffers I'd normally make local variables pre-allocate them, but I'd prefer not to, and it doesn't solve the allocation occuring in something like std::string.substr(). Any suggestions? Do the allocators already minimize this problem? Is there something I'm missing? Are my concerns even valid? (Translation: Help Please :) ).
Advertisement
The first thing to do would be to measure the level of fragmentation and see if it really is a problem. If you can't do this, but you have an idea of your programs allocation patterns, you can easily simulate this.

If fragmentation is a problem, one possible solution is to replace the heap manager with one that utilizes separate areas for blocks of different sizes.
Quote:Original post by SnotBob
If fragmentation is a problem, one possible solution is to replace the heap manager with one that utilizes separate areas for blocks of different sizes.


A heap manager is a good solution to this problem, if it really is a problem.

Quote:Original post by linternet
I work on a high availability application


Does this mean the program is portable across platforms?
If it is a Windows program you can look into the functions HeapCreate() and HeapSetInformation() using the LFH Flag.
More information for windows specific information can be found here:
http://msdn.microsoft.com/en-us/library/aa366750(VS.85).aspx
As of Vista, the Low Fragmentation heap allocator is used by default

I regularly work on a high-availability C++ Server app that runs on Windows and is part of our Access Control system. It's used in mines etc where it can be life threatening if the server goes down.
Basically we make plenty of use of the STL and never bother with custom allocators. Eventually if the server does crash then it's always something else like a deadlock, or a stray tiny memory leak, or hard disk failure, or something like that, and never a fragmentation problem. So I would not worry about it.
"In order to understand recursion, you must first understand recursion."
My website dedicated to sorting algorithms
@iMalc:

I wouldn't run windows on the server if it is lifethreatening when the server goes down.
Someone who uses a, euhm..., delta!?
Is fragmentation really an issue?

On the PC, nearly every operating system out there uses virtual memory. You won't start seeing fragmentation unless you routinely keep a gigabyte or so of memory in use, and routinely request very large chunks of memory.

Generally on the PC you will have other memory management issues long before memory fragmentation is a problem. You will be using custom heaps and custom allocators, complex resource managers, and be listening to os notifications about memory, and those will generally all take place before fragmentation is an issue.


But if you are not on a PC, then fragmentation is such a big issue you should have policies in place. Consoles, handhelds, and other devices do not have the luxury of virtual memory, and fragmentation is generally a concern before the first line of code is written.
Quote:Original post by delta user
@iMalc:

I wouldn't run windows on the server if it is lifethreatening when the server goes down.


Don't know about Vista, but from my experience, XP with the service packs is rock stable.

This topic is closed to new replies.

Advertisement