Sign in to follow this  
Animeplayer

Creating a Memory Pool

Recommended Posts

Animeplayer    100
Hello Everyone, it's been a long time since I posted up something. I was wondering if anyone could provide a tutorial on Memory Pool, I have been having a hard time finding any type of resource to learn on how to create my own. I understand why you cannot use memory dynamically allocated commands in the C program i.e.(new and malloc) because they are not as efficient as creating your very own. So can anyone provide me with some materials or tutorial as to how to create my own?

Share this post


Link to post
Share on other sites
rip-off    10979
[quote]
I understand why you cannot use memory dynamically allocated commands in the C program i.e.(new and malloc) because they are not as efficient as creating your very own.
[/quote]
Have you determined where you need to pool memory? This is an optimisation, and thus must be carefully and selectively applied. If you try to pool everything you end up with a general memory allocator, and likely not particularly faster than the runtime's version.

What are your requirements? Memory pools work best when they are used with a fixed size, this is the easiest one to create and manage, and avoids fragmentation.
[quote]
So can anyone provide me with some materials or tutorial as to how to create my own?
[/quote]
I typed "Memory Pool Tutorial" into Google and get lots of results. I don't have time to screen them, but can you address why you found them deficient?

Share this post


Link to post
Share on other sites
jyk    2094
[quote name='Animeplayer' timestamp='1307614912' post='4821258']
I understand why you cannot use memory dynamically allocated commands in the C program i.e.(new and malloc) because they are not as efficient as creating your very own.[/quote]
Certainly you can use malloc (in C and C++) and new (in C++) in your programs. In fact, that's probably exactly what you'll want to do unless you have a specific need for a custom solution. (Even then, you might look into existing solutions, such as boost::pool if you're using C++, rather than implementing something from scratch.)

[Edit: Fixed error in post.]

Share this post


Link to post
Share on other sites
Hodgman    51328
A not very efficient, but simple pool:[source lang=cpp]class MyPool
{
private:
MyObject objects[9000];
bool isUsed[9000];
public:
MyPool()
{
for(int i=0; i<9000; ++i)
isUsed[i] = false;
}
MyObject* Alloc()
{
MyObject* result = NULL;
for(int i=0; i<9000; ++i)
{
if(isUsed[i] == false)
{
isUsed[i] = true;
result = &MyObject[i];
}
}
return result;
}
void Free(MyObject* ptr)
{
assert( ptr >= objects && ptr < objects+9000 );
int index = (int)(ptr - objects);
assert( isUsed[index] == true );
isUsed = false;
}
};[/source]This kind of thing was everywhere in old C game code-bases... What's made you decide you need one?

Share this post


Link to post
Share on other sites
Animeplayer    100
[quote name='rip-off' timestamp='1307616756' post='4821264']
[quote]
Have you determined where you need to pool memory? This is an optimisation, and thus must be carefully and selectively applied. If you try to pool everything you end up with a general memory allocator, and likely not particularly faster than the runtime's version.

What are your requirements? Memory pools work best when they are used with a fixed size, this is the easiest one to create and manage, and avoids fragmentation.
[/quote]
Alright, let's say I want to use 65536 in my memory pool and allocating from sizes of 2 bytes to 16k bytes.

[quote]
I typed "Memory Pool Tutorial" into Google and get lots of results. I don't have time to screen them, but can you address why you found them deficient?
[/quote]
The ones I came up with was more explanation about memory pool but could not find any comparison of codes between creating your own without using Dynamic allocated commands in C that is given.
[/quote]

Share this post


Link to post
Share on other sites
Animeplayer    100
[quote name='Hodgman' timestamp='1307621021' post='4821274']
This kind of thing was everywhere in old C game code-bases... What's made you decide you need one?
[/quote]
I was asked this question to code my own pool, I did not know how to approach it since my college didn't teach me this. So I knew that this is a new skill I must learn.

Share this post


Link to post
Share on other sites
kilah    514
As far as I am concern, both malloc and new create fragmented memory on most OS and are not particulary optimal in that sense. That doesn't mean you need to replace them, means you need to be aware on this fact.

Share this post


Link to post
Share on other sites
SiCrane    11839
malloc() and new aren't OS specific, they're compiler and standard library implementation specific. For that matter, the behavior of those can be dependent on things like environment variables and whether or not the the program is launched from a debugger. For instance, even in release mode, if you launch an MSVC project from the IDE it'll use an allocator that has more bookkeeping information, but some Borland compilers on the same OS will already do memory pooling for small block allocations.

Share this post


Link to post
Share on other sites
rip-off    10979
[quote]
I did not know how to approach it since my college didn't teach me this.
[/quote]
College will not teach you everything. As I mentioned earlier, I get lots of results when searching Google for this topic. If you're having tourble, can you ask a more specific question. E.g. "I was reading [url="http://www.google.com/search?q=memory+pool"]here[/url] but I don't understand concept X or code fragment Y."

Share this post


Link to post
Share on other sites
kilah    514
You are correct compilers use their own tricks to try to have more efficient dynamic memory pooling, yet in the end it has to rely on a sys_call. It was incorrect from me to name it OS dependant.

Share this post


Link to post
Share on other sites
rip-off    10979
[b]All[/b] memory allocators rely on a system call (at some level) - unless they want to restrict themselves to the memory available on the stack. I'm not sure what point you are making?

Share this post


Link to post
Share on other sites
PrestoChung    334
[quote name='kilah' timestamp='1307663918' post='4821534']
As far as I am concern, both malloc and new create fragmented memory on most OS and are not particulary optimal in that sense. That doesn't mean you need to replace them, means you need to be aware on this fact.
[/quote]


The fragmentation must be larger than cache-line scale, correct? Where is the slowdown or the mechanisms that this fragmentation affects?

Share this post


Link to post
Share on other sites
kilah    514
@rip-off: Nothing I was just stating he is 100% right on what he pointed out.

@PrestoChung: Memory fragmentation may affect specially on systems that do not have ""virtual memory"" (most likely embedded systems). For instance a pretty simple example, would be allocating 4 blocks of 4MB memory on a 16MB system (asssuming we can use it all), if we free 2 of those block and those are not contiguous, you will not be able to allocate 8MB block, yet you have 8MB free memory in your system. Quite simplified problem but that's the point I was trying to make (and it seems i did not explain myself correctly).

Hopefully this clarifies a bit, my point.
Cheer.

Share this post


Link to post
Share on other sites
Animeplayer    100
[quote name='rip-off' timestamp='1307698442' post='4821635']
[quote]
I did not know how to approach it since my college didn't teach me this.
[/quote]
College will not teach you everything. As I mentioned earlier, I get lots of results when searching Google for this topic. If you're having tourble, can you ask a more specific question. E.g. "I was reading [url="http://www.google.com/search?q=memory+pool"]here[/url] but I don't understand concept X or code fragment Y."
[/quote]
Yeah, but I want to gain access to those missing skill I want to know, especially toward game development. Sorry, I tend to confuse people by accident, I meant that, whenever I looked online about Memory Pool, I kept finding Source Codes that involved using dynamic allocations. Hodgman provided a example of a memory pool source code that I was looking for that didn't use dynamic allocations.

@Hodgman Thanks, this was the example I was looking for, just need the allocation and de-allocation of that memory pool.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this