Jump to content
  • Advertisement
Sign in to follow this  
Ysaneya

Lib for multithreaded memory alloc

This topic is 4865 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I'm looking for a memory allocation library, with these properties: 1. Must solve the heap contention problem when used in a multithreaded environment. 2. Must be portable (Linux and Windows at least, then more is better). 3. Must be fast. 4. Must be under the public domain or a LGPL-type license (allowed for commercial usage). 5. (If possible) Must be transparent (no need to modify your source code). The library "hoard" so far looked like the best candidate but it's under the GPL :( Any suggestion is welcome.. Y.

Share this post


Link to post
Share on other sites
Advertisement
Check out ptmalloc2.

To truly solve #1, you need a lock-free allocator. I happen to be working on one off and on (when time permits), but it's a long way from done :/

// edit:
Out of curiosity, what are you going to use it for?

Share this post


Link to post
Share on other sites
Blizzard uses a fairly simple system: have 256 separate heaps to allocate from, each protected by a critical section, and arranged in a hash table based on the filename of the source file that calls SMemAlloc. Using the filename doesn't seem like such a good idea. Line number would have been better. But if evenly distributed, 256 heaps is substantially more than enough for multithreaded programs.

Share this post


Link to post
Share on other sites
256 and select by source file? Interesting. I guess they didn't have TLS then, or else per-thread heaps would've been easier.
But eventually, even those must serialize on a global heap from which to grow them. Sure, you can limit allocations to a certain amount or set aside parts of the address space per-thread, but that's not a general solution.
Therefore, a completely lock-free allocator is necessary :)

Share this post


Link to post
Share on other sites
Thanks for the answers.

I'm using a per-thread heap allocator at the moment, but it looks like a hack, so i'm not satisfied with it.

Ptmalloc2 looks good, but i'm having troubles to compile it under MSVC. Tomorrow i'll try to install GCC and see if it behaves better. I haven't found an easy to use DLL on google :(

I'm working on some procedural textures generation, and given the complexity and resolution of these, it can take minutes to calculate. It has to be done perfectly in parallel with my rendering loop. Preallocating the data is impossible due to its dynamic nature.

Y.

Share this post


Link to post
Share on other sites
Okay, i've successfully compiled ptmalloc2 as a DLL under Win32.

It works very well so far. Not only it fixed my heap contention problem in multithreading, but it's also a lot faster than the standard malloc in normal (non-threaded) applications. I've measured an average of 4 to 5 times performance increase!

-> Back to more testing.

Y.

Share this post


Link to post
Share on other sites
Glad to.
Ah, thanks for the infos; sounds tricky.

Excellent! :) Wow, that's quite a difference. VC's malloc is known to be slow but has improved since VC5. Could it be that you have an old msvcrt.dll on the path? (this is no longer an issue from VC.net onwards because they renamed the DLL)
Man, I wish I had more time for the lock-free allocator; it'd be very interesting to see how it would measure up here.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!