Problem defining a new 'new' macro

Started by
7 comments, last by Aardvajk 10 years, 2 months ago

I'm trying to implement a crude memory tracking, following this source:

http://www.chrisculy.com/writings/Programmatic_Memory_Leak_Detection_in_C++.pdf

However, most things works fine (apart from delete crashing) except that I can't macro the new keyword. I'v been searching far and wide for any answers on how to do it. But I haven't found anything.

I'm using Visual Studio 2012, here is the source:

Rememrall.h


#ifdef _DEBUG
void* operator new(size_t size, const char* file, unsigned int line);
void operator delete(void* memory);

#undef new
#define new new(__FILE__, __LINE__)

#endif

void reportMemoryStatus();

Remembrall.cpp


#ifdef _DEBUG
#include "Remembrall.h"
#include "Logger.h"
#include <cstring>
#include <cstdlib>
#include <list>

#include <iostream>


static std::string stripped_path( const char* str ) 
{
	std::string path = str;
	return path.substr(path.find_last_of('\\')+1, path.size()-1);
}

struct MemoryRecord
{
	const char* file;
	unsigned int line;
	unsigned int size;
	unsigned long address;
};

static std::list< MemoryRecord > memoryArchive;


void* operator new(size_t size, const char* file, unsigned int line)
{
	void* memory = malloc(size);

	MemoryRecord record;
	record.file = file;
	record.line = line;
	record.size = size;
	record.address = (unsigned long) memory;

	memoryArchive.push_back(record);

	return memory;
}

void operator delete(void* memory)
{
	/*
	auto it = memoryArchive.begin();
	auto end = memoryArchive.end();


	while(it != end)
	{
	if( (*it).address == ( unsigned long ) memory )
	{
	memoryArchive.erase(it);
	break;
	}
	it++;
	}
	*/

	free(memory);
}




void reportMemoryStatus()
{
	auto it = memoryArchive.begin();
	std::string info = "";
	while( !memoryArchive.empty() )
	{
		info = "[MEMLEAK]\tFile: ";
		info += stripped_path((*it).file);
		info += "\tLine: ";
		info += std::to_string((*it).line);
		info += "\tAddress: ";
		info += std::to_string((*it).address);
		info += "\tSize: ";
		info += std::to_string((*it).size);
		std::cout << info << std::endl;
		db::logger.write( info, db::outs::error, true );
		memoryArchive.erase(it);
		it = memoryArchive.begin();
	}
	db::logger.panic();
}
#else
void reportMemoryStatus() {}
#endif

EDIT

I found a solution, making a new header, include the Remembrall-header then AFTER that add the new macro. It feels kind of hackish, is this really the only way?

Advertisement

http://wyw.dcweb.cn/leakage.htm walks through the whole process and includes source code. He covers more than my last employer's solution covered, and my last employer is a $1+ billion company. It's possible he missed something, but it sure seems top notch.

That is a great resource, but it doesn't explain my problem unfortunatly :/

To be blunt, this is just a silly way to do things these days.

Even if you get it to work, it'll half the time just give you the file/line of some template code and won't be remotely helpful in tracking down a lot of allocations.

You can rather efficiently look up the callstack and get the return address of the function calling new. Store that. You can take it a step further and store the addresses of the last 5 (or more) call frames. During debug dump commands, you can use the symbol database functions (they differ between OS) to make those address to the file, line, and function name they're located at. All only in debug builds, naturally.

As a bonus, this works with proper C++11 idioms where you barely ever even call operator new yourself.

This can be further combined with a memory context system. This would allow you to not just track that some allocation came from a particular file but allow you to easily print in-game memory reports like seeing which allocations are graphics-related or physics-related or whatnot without needing to make every allocation call tagged. Something like:

void render(... stuff ...) {
  MemoryScope("Graphics", [&]{
    // any allocations in this block or any function called by this block are now tagged as belonging to Graphics,
    // no matter what file they're in, unless another MemoryScope is entered.
  });
}
You can use a macro and a scoped local variable if you prefer that syntax over the lambda one. I prefer the explicitness, but that's just me.

Sean Middleditch – Game Systems Engineer – Join my team!

To be blunt, this is just a silly way to do things these days.

Even if you get it to work, it'll half the time just give you the file/line of some template code and won't be remotely helpful in tracking down a lot of allocations.

You can rather efficiently look up the callstack and get the return address of the function calling new. Store that. You can take it a step further and store the addresses of the last 5 (or more) call frames. During debug dump commands, you can use the symbol database functions (they differ between OS) to make those address to the file, line, and function name they're located at. All only in debug builds, naturally.

As a bonus, this works with proper C++11 idioms where you barely ever even call operator new yourself.

This can be further combined with a memory context system. This would allow you to not just track that some allocation came from a particular file but allow you to easily print in-game memory reports like seeing which allocations are graphics-related or physics-related or whatnot without needing to make every allocation call tagged. Something like:


void render(... stuff ...) {
  MemoryScope("Graphics", [&]{
    // any allocations in this block or any function called by this block are now tagged as belonging to Graphics,
    // no matter what file they're in, unless another MemoryScope is entered.
  });
}
You can use a macro and a scoped local variable if you prefer that syntax over the lambda one. I prefer the explicitness, but that's just me.

I'm sure there is tons of better ways, and I do appreciate the effort in showing me another way. However I'm not looking for an alternative way right now, this is for a small project on a deadline. Also I'd love to actually know why this isn't working!

So far I have gotten it to work in two ways:

  • Write all the code inside the header and add #define new new(__FILE__,__LINE__), this however dosen't work if two headerfiles includes the memoryleak header even if it's inside a header-guard. The compiler compains of already used function symbols.
  • Sepperate the leak code into a .h and .cpp, wrap it around another header and add the macro last.

I'm trying to figure out WHY my first attempt isn't working, and how to "properly" do it.

You shouldn't write code in a header file. Of course that causes duplicate symbols. I didn't follow what you're doing in the second point there.

Sorry, but I don't think your approach can be fixed. It is likely already relying on undefined behavior. You are using "#define new new(__FILE__,__LINE__)" which means your "void* operator new(size_t size, const char* file, unsigned int line)" becomes "void* operator new(__FILE__,__LINE__)(size_t size, const char* file, unsigned int line)" which doesn't even make sense. For your approach to work, you'd need the #define to be simultaneously visible and not visible. Short of a separately linked library, I don't see it happening. This is why you always see:

#define DEBUG_NEW new

#define new DEBUG_NEW(__FILE__,__LINE__)

Why are you opposed to that approach? Maybe if you explain the reason it's not a viable solution, we can help you fix the problem.

If you want a different approach: http://blogs.msdn.com/b/calvin_hsia/archive/2009/01/19/9341632.aspx uses ebp (first level of call stack) and overrides instead of #define.

Since you write that you are on VS2012 and also that you are under a deadline, the effort of writing and debugging your own crude tracker that only works in limited cases doesn't make much sense.

Microsoft's runtime libraries include many useful tools, including the debug memory heap. Suggested reading. You get access to much of it simply by using the _DEBUG compile time option, which is the default for Visual Studio's debug builds. A few of the features require a tiny bit of extra work, but it is both debugged and documented, making it generally better than fumbling your way through by experimentation.

There are some more comprehensive external libraries for memory tracking, but they usually require much more work, including re-compiling all your linked-to libraries from source and other burdensome tasks.

Since you write that you are on VS2012 and also that you are under a deadline, the effort of writing and debugging your own crude tracker that only works in limited cases doesn't make much sense.

Microsoft's runtime libraries include many useful tools, including the debug memory heap. Suggested reading. You get access to much of it simply by using the _DEBUG compile time option, which is the default for Visual Studio's debug builds. A few of the features require a tiny bit of extra work, but it is both debugged and documented, making it generally better than fumbling your way through by experimentation.

There are some more comprehensive external libraries for memory tracking, but they usually require much more work, including re-compiling all your linked-to libraries from source and other burdensome tasks.

I know there are tons of better, more well established methods. In this case I wanted to learn something. I'm not to found of getting tons of suggestions telling me I can avoid learning anything by just using a library. Don't get me wrong, I appreciate the effort of others, but when everyone says; "Just use <insert lib> instead", It get on my nerves.

However, I got my code to work under circumstances the only problem was I didn't know exactly why it worked or why it didn't work. I actually got the code to work as intended with a single .h and .cpp file.

I put the "#define new new(__FILE__,__LINE__) at the end of the header and right after all the includes in my .cpp file I added #undef new
I haven't ran into any problems with this yet and I'm quite happy with the result.

In the future I guess I should make it clear from the very beginning if I'm interested in alternative solutions or not, in this case I wanted to understand the raw code of it all.

#define DEBUG_NEW new

#define new DEBUG_NEW(__FILE__,__LINE__)

Most articles had something similar:


#define DEBUG_NEW new(__FILE__,__LINE__)
#define new DEBUG_NEW

This didn't even remotely work for some reason.

In this case I wanted to learn something.

I'll get on your nerves too then and point out that a project on a deadline is not the place to be learning new things unless absolutely necessary.

This topic is closed to new replies.

Advertisement