I've got a quick question about monitoring memory use in C, before I go hog-wild with my own solutions. I've written wrappers for memory allocation and deallocation in my code, with the same interface as for the standard C library:

void *dgl_malloc(size_t size);
void dgl_free(void *pointer);

(DGL is the prefix I'm using for my code. I'm not using calloc, so I skipped it) These are just wrappers for the standard library functions at the moment. I'm more interested in logging memory use than writing my own allocators. The problem is that there's not a lot I can do with those void pointers. Say for example I want to keep tabs on the amount of memory my functions have allocated. I can increment a counter in dgl_malloc based on the size requested. But when it comes time to release the memory there's no info in that pointer provided to dgl_free to decrement it. I don't think I can do crazy things with structures, such as making something like:

typedef struct {
void *memPointer;
unsigned long size;
} memStructure;

then creating one of these in dgl_malloc, returning memPointer, then back-casting to memStructure in dgl_free. If I allocate the pointer to another variable, then the extra info is lost. Is there a simple solution to this that I'm overlooking, or do I have to do hacky? (Note I'm using C, not C++, so C++ only functionality can't be used)

Which compiler(s) are you using? Your C standard library probably already provides functions for getting the size from a pointer. Ex: _msize() with MSVC or malloc_usable_size() with GNU's standard C library implementation.

Quote:Original post by SiCrane
Which compiler(s) are you using? Your C standard library probably already provides functions for getting the size from a pointer. Ex: _msize() with MSVC or malloc_usable_size() with GNU's standard C library implementation.

I use GCC (I'm using Xcode on a Mac). I'm generally rather wary of compiler specific functions because I like portability, but if it's the only easy way to solve the problem I'll make an exception.
Edit: although thinking it through, it's probably better in this case because the system functions return the actual memory used, not the allocated size. It's a somewhat small difference given I'm mainly doing this to spot potential memory leaks, but it helps.

It would only work for recording memory sizes. That's pretty much all I want to do at the moment, but if for some reason I wanted to store extra info (time of allocation, maybe) I'd need some way of storing meta-info with the pointer. Is there a method of doing that via some kind of struct while still returning a (void *) just like malloc? Or is this not poisslbe and I would have to use a separate method of recording information, like keeping records of each pointer in a list or tree?

If you wanted to store it yourself you could do something like:

void * my_malloc(size_t size) {  size_t * buffer = malloc(size + sizeof(size_t));  *buffer = size;  return buffer + 1;}

Then for free you'd need to subtract the size of size_t to get back the original pointer. Be careful that you may no longer meet alignment requirements if size_t isn't the same size as the largest primitive.

Ah, that would work. I'm not sure why I didn't think of that.

I'm not that well versed on alignment requirements (I've always treated memory as a sort of magic thing the compiler deals with). Is there an easy way to ensure that extra info is well aligned? I could always stick it in a union with some padded value, if there's a way of determinine the size of what that value should be.

This is pretty much your best solution short of writing a custom allocator. You mention allocation requirements, as you correctly point out alignment can become a problem. Not in the sense that it will not even work, but in the sense that if the allocator you're wrapping guarantees an alignment of N, then the only way you can maintain this guarantee *for the pointer the user sees* is if you allocate a multiple of N extra bytes up front. This is probably not a big deal for your purposes since you're probably only concerned with aligning on multiples of 8 or 16. I was once faced with a situation where I needed my memory blocks to be aligned on boundaries of 4096, and in this case it was a real problem because I could not allocate an additional 4KB of memory every time.

I just saw this:

Quote:Original post by Trapper Zoid
It's a somewhat small difference given I'm mainly doing this to spot potential memory leaks, but it helps.

In that case, you might want to look at your standard library's memory debugging functions. For glibc you've got mprobe(), mtrace(), __malloc_hook, __free_hook, __malloc_initialize_hook and malloc_usable_size(). And probably some others I'm forgetting about.

Quote:Original post by Trapper Zoid
I could always stick it in a union with some padded value, if there's a way of determinine the size of what that value should be.

For the most portable code, you can create a union of all the primitive types as will as some basic derived types like pointers and use the size of that as a the alignment value. If your platform supports it, you could use some alignof keyword on that union (usually spelled __alignof or __alignof__). However, if you're using platform specific code, you might as well just determine empirically what the alignment of the underlying allocator is and use that.

Quote:Original post by cache_hit
I was once faced with a situation where I needed my memory blocks to be aligned on boundaries of 4096, and in this case it was a real problem because I could not allocate an additional 4KB of memory every time.

Again, platform specific code can be your friend here. Ex: glibc's valloc() function which always aligns allocations on a page boundary.

Quote:Original post by SiCrane

Quote:Original post by cache_hit
I was once faced with a situation where I needed my memory blocks to be aligned on boundaries of 4096, and in this case it was a real problem because I could not allocate an additional 4KB of memory every time.

Again, platform specific code can be your friend here. Ex: glibc's valloc() function which always aligns allocations on a page boundary.

Yea in that situation though I was going out of my way to use the allocator provided by Intel TBB, since it's the fastest multi-threaded allocator there is. Ultimately I ended up not being able to use it, but oh well.