Jump to content
  • Advertisement
Sign in to follow this  
alnite

so, tell me about STL causing memory fragmentation again?

This topic is 4501 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I have heard occasionally of STL causing memory fragmentation (here in Gamedev and other websites). I thought it was just bad practice of STL, but now I am curious if there is any proof of it? As a matter of fact, STL has Allocator that's supposed to manage it.

Share this post


Link to post
Share on other sites
Advertisement
As far as I understand it, most typical implementations of c++ implement the standard allocators in terms of ::operator new() and ::operator delete(), which are usually implemented in terms of malloc() and free(), which aren't really all that efficient at preventing memory fragmentation.

Share this post


Link to post
Share on other sites
The stl allocator is designed on the principle that you can replace it with something fitting to the situation.
But be aware that there is no fit-all solution to memory management. Depending on what you are storing in an STL container, one allocation type will be
better than another.

The most often brought up argument is when someone is using a STL container to collect many small objects that spawn and die rapidly.
The standard allocator is usually very poor at this, and fragments memory very quickly in the process.
boost::pool_allocator or something similar is what you are probably looking for as far as reducing fragmentation if you are thrashing the data
in a STL container in this fashion.

Share this post


Link to post
Share on other sites
Are new and delete implemented in terms of malloc and free, or are they implemented in the same way malloc and free are implemented i.e. os calls?

I would have thought that, under the hood, a compiler could implement default new and delete in any way that was most efficient for the platform.

My understanding was that library code could be implemented in assembler if necessary to give maximum performance. That was why, in the good old C days, one should prefer strcpy (for example) to your own while(*a++=*b++); implementation, since the library would probably be taking advantage of machine-level speed ups.

Could be wrong.

Share this post


Link to post
Share on other sites
Quote:
Original post by EasilyConfused
Are new and delete implemented in terms of malloc and free, or are they implemented in the same way malloc and free are implemented i.e. os calls?

I would have thought that, under the hood, a compiler could implement default new and delete in any way that was most efficient for the platform.

My understanding was that library code could be implemented in assembler if necessary to give maximum performance. That was why, in the good old C days, one should prefer strcpy (for example) to your own while(*a++=*b++); implementation, since the library would probably be taking advantage of machine-level speed ups.

Could be wrong.


Memory allocation has little to gain from low-level optimizations: both the C and C++ interfaces have to read and update an index of allocated memory, without any CPU-intensive activity.
There can be, in theory, algorithmic optimizations (cheaper and smaller index data structures), but most language runtime libraries are already mature and competently written.
Memory fragmentation doesn't result from bad library code, but from the use of general purpose policies: not knowing how long a requested memory block is going to remain alive, the allocation system can do no better than put it in the first available place. Only a custom allocator or object pool can make smarter decisions.
The Java virtual machine has the same problem, but current implementations have compacting garbage collection: live objects are transparently moved around in memory to merge the gaps and to separate long-lived and short-lived objects.
This sophistication is sadly impossible in C or C++.

Share this post


Link to post
Share on other sites
Quote:
I have heard occasionally of STL causing memory fragmentation (here in Gamedev and other websites).


Any time you allocate memory in increasingly larger chunks, releasing the old chunk after reallocation, you risk fragmentation. There is no way around that behaviour if you ever need to make your vectors and strings grow. It's not a STL-specific problem, but a problem with any container that provides the guarantees vector and string must. For example, deque won't have this problem, but you lose the contiguous storage guarantee.

Quote:
Are new and delete implemented in terms of malloc and free, or are they implemented in the same way malloc and free are implemented i.e. os calls?


It's implementation-defined. Compiler and standard library writers are free to implement them as they wish.

Quote:
This sophistication is sadly impossible in C or C++.


Live objects might not be moveable, but GCs are possible.

Share this post


Link to post
Share on other sites
Quote:
Original post by Fruny
Live objects might not be moveable, but GCs are possible.

Garbage collection doesn't help against fragmentation: once a C or C++ object is allocated, it remains pinned at a fixed address no matter how it is later deallocated.
Pointers contain the actual address of objects and not, like in typical JVM implementations, the address of a system-managed "handle" containing the memory address.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!