Questions about memory management and operator new in game engine design.
Author
Hello,
I have questions about memory management and operator new in game engine design.
Ok, so I know real time performance is critical and that operators new/delete are slow. I have a memory allocator class that I wrote that uses free lists and paging to simulate allocation of objects and avoids allocations except when new pages are required.
Now, I know that C++ stl containers by default use operators new/delete for their allocators. My engine uses stl in quite a lot of places within the engine, including resource instancing, sorting objects, and most data that requires containers are using stl containers so that means a lot of new/delete calls.
I have an idea that I would like some input on.
What I would like to do is to design a memory manager module that simply has a bunch of memory allocators which distribute pointers to memory of different sizes (see above). Next, overload the global operator new. Within the overloaded new operator, it should check the size of the object requiring allocation and then find the appropriate allocator (the one that allocates memory of slightly larger size) and return the pointer.
The idea then is that EVERY new/delete in the program (including stl I believe) is handled through the memory manager system via operator new. This would greatly improve performance at runtime I believe, and while there may be some extra memory consumption it should be acceptible as it is my understanding that memory is fairly cheap these days.
Any input?
If this doesn't work as I would like, please leave some input. And if it does not, please suggest alternatives.
It just seems a hassle to have to plug in your own allocators everywhere that is performance critical and handle allocations of stl in that manner when an overloaded operator new with a memory manager module should just take care of it everwhere and more efficiently.
Thank you for your help.
Jeremy
With the STL containers taking an allocator as a parameter.
You are better off just giving allocators to your containers. It will give you better control over the memory management.
Put memory pool allocators on containers you allocate a lot with.
And for containers you allocate once at run time (or once per level) make a more simplistic allocator that only has to carve up one block into pieces, and when the level is done you just deallocate the entire block instead of the pieces. Or you can make sure a block of memory is aligned or padded for specific needs (cache lines, SSE instructions, etc)
Here are a couple gamedev links to get you started: Here and Here
Other things to note.
It is nice to know the file/line of an allocation.
It is nice to know the TYPE of allocation. (object type)
It is nice to know the ASSET TYPE of allocation. (debug, collision, audio, graphics, etc)
Having different allocators for each class and each type (while it may be a pain to plug in the first time) can be really helpful in diagnosing memory problems (as well as letting you know where your memory budget went. and on a console, memory budgets get really tight, so asset tracking is very helpful)
You are better off just giving allocators to your containers. It will give you better control over the memory management.
Put memory pool allocators on containers you allocate a lot with.
And for containers you allocate once at run time (or once per level) make a more simplistic allocator that only has to carve up one block into pieces, and when the level is done you just deallocate the entire block instead of the pieces. Or you can make sure a block of memory is aligned or padded for specific needs (cache lines, SSE instructions, etc)
Here are a couple gamedev links to get you started: Here and Here
Other things to note.
It is nice to know the file/line of an allocation.
It is nice to know the TYPE of allocation. (object type)
It is nice to know the ASSET TYPE of allocation. (debug, collision, audio, graphics, etc)
Having different allocators for each class and each type (while it may be a pain to plug in the first time) can be really helpful in diagnosing memory problems (as well as letting you know where your memory budget went. and on a console, memory budgets get really tight, so asset tracking is very helpful)
Quote:I know [...] that operators new/delete are slow.
How do you know it? Did you benchmark it?
Quote:Original post by loufoqueQuote:I know [...] that operators new/delete are slow.
How do you know it? Did you benchmark it?
I think what he meant to say was that dynamic memory allocation/deallocation is slow, if he doesn't then he is probably miss-interpreting the problem.
As I would look at; new/delete([]) are for allocation of dynamic (heap) memory, and yes it is slow, thats a given no need for profiling. If you want to minimise said memory allocation why not create some sort of object manager where a maximum amount of object storage is set aside and you use these pre allocated objects when required and available, of course you would need to account for the case where there is no free objects available, maybe a 'new' call. This strategy would also appear to fit well with the pimpl idiom, pointer to implementation, allowing you to more efficently change the implementation of a particular object instance, and if your pimpl's had some redefinition functionality calls to new would be very infrequent.
I have yet to really get my teeth in to this area so there could be huge holes in what I have outlined here.
Quote:Original post by grill8
The idea then is that EVERY new/delete in the program (including stl I believe) is handled through the memory manager system via operator new. This would greatly improve performance at runtime I believe, and while there may be some extra memory consumption it should be acceptible as it is my understanding that memory is fairly cheap these days.
IMO, it would be better if you just use resize/reserve on STL containers when it's appropriate. You will gain a tremendous performance boost, if you use those functions wisely. As well as using the right STL container can have a huge performance boost:
I wrote a memory leak detector and used a vector or a map to keep track of the allocations. With some simple programm that allocated some thousand arrays of constant size, the vector implementation took about 50 seconds, whereas the map took about 4 seconds.
I really doubt that you can make the STL work faster by writing your own memory manager (as far as the STL implementation from microsoft is concerned, I haven't used other implementations very often). Their implementation is pretty clever, as it preallocates more memory than you need in the first place. Though it's more efficient to preallocated the things you already know you are going to need (eg. you know there are 50 meshes in your level, so reserve that many memory for your containers).
You also have to ask yourself if the rest of your allocations have a great impact on your code, and if you can prevent that somehow.
Quote:Original post by grill8
I have questions about memory management and operator new in game engine design.
Generally speaking, implementation details should not affect design.
Quote:
Ok, so I know real time performance is critical and that operators new/delete are slow. I have a memory allocator class that I wrote that uses free lists and paging to simulate allocation of objects and avoids allocations except when new pages are required.
A program is slow where it takes too long to perform an action. You determine this through measurement. Identify the slow parts through measurement, then make an effort to speed those parts up.
Do not assume you need to put a lot of effort into speeding something up until you have identified it as a slow bit.
Quote:
Now, I know that C++ stl containers by default use operators new/delete for their allocators. My engine uses stl in quite a lot of places within the engine, including resource instancing, sorting objects, and most data that requires containers are using stl containers so that means a lot of new/delete calls.
The container found in the C++ standard library use their allocator member to allocate their memory. By default, the standard containers use std::allocator as their allocator, and std::allocator is specified as "allocating as if using ::operator new()". What that means in practice is that at some point, std::allocator uses ::operator new() to allocate some memory, but most implementations use a slab allocation strategy with pooling.
Often, your standard library implementation will provide alternative allocator implementations, and there are third-party add-on allocators (like from boost) that do fany things.
Fact is, until you have identified where the performance bottlenecks are in your program (through appropriate testing and merasurement) you should stick with the default and focus on functionality. Make it work, then make it work fast.
Quote:Fact is, until you have identified where the performance bottlenecks are in your program (through appropriate testing and merasurement) you should stick with the default and focus on functionality. Make it work, then make it work fast.
I disagree and I think this default answer is given out far too often here.
Consider a situation where I know my program will allocate millions of small objects during it's lifetime. It is plainly obvious that having memory allocated at program start and placing the objects into that memory will be faster than dynamically allocating the objects when needed. I don't need to test performance, I already know it will be faster, and even if it isn't the major bottleneck of the program, optimising early will still give performance gains.
Make it work fast, then make it work faster...
Quote:Original post by trippytarka
I disagree and I think this default answer is given out far too often here.
...
I think what Begma means is that he should not introduce some fancy memory allocation factory unless he finds out that the way the STL manages memory is a bottleneck. The standard functionality of the STL already grants the ability to reserve memory before needed as I described it and you pointed out with that example.
In simple terms: use the STL properly and a custom factory won't be needed in 99% of the cases.
The first thing most console games do is replace the CRT allocator by overriding malloc and new. As far as I know about STL, I don't use it in the games I work on, the STL uses operator new for some of its containers even when you provide a custom allocator. You can read up on EASTL more if you want to know the reasons why.
I'm going to go against most people's advice on this thread and say that thinking about memory management, writing custom allocators, and overriding new/malloc from the beginning is going to be a huge benefit on performance for your project. In my experience whenever a system didn't think about allocation patterns the performance implications hit it towards alpha/beta. Though, the suggestions on reserving memory for your containers is great advice! Pools of memory will always help out.
As far as writing your own allocator I would recommend checking out Doug Lee's memory allocator (dlmalloc) http://g.oswego.edu/dl/html/malloc.html . It's the industry's verbal agreement on the best general purpose allocator out there and is used in numerous titles.
Good luck!
-= Dave
I'm going to go against most people's advice on this thread and say that thinking about memory management, writing custom allocators, and overriding new/malloc from the beginning is going to be a huge benefit on performance for your project. In my experience whenever a system didn't think about allocation patterns the performance implications hit it towards alpha/beta. Though, the suggestions on reserving memory for your containers is great advice! Pools of memory will always help out.
As far as writing your own allocator I would recommend checking out Doug Lee's memory allocator (dlmalloc) http://g.oswego.edu/dl/html/malloc.html . It's the industry's verbal agreement on the best general purpose allocator out there and is used in numerous titles.
Good luck!
-= Dave
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
