• Advertisement
Sign in to follow this  

how new works (C++)

This topic is 4281 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

i'm looking at making a simple memory checker to improve my knowledge of the language, but i can't find any good doc's (the msdn tells me how to use it but not how it works) on how the new operator and variants function so i don't know how to implement my own debug version. does anyone know of any good docs, or have any other suggestions? thanks

Share this post


Link to post
Share on other sites
Advertisement
The default new usually calls malloc() to allocate memory for the object, then it calls the object's constructor. There isn't any more to it than that.

Share this post


Link to post
Share on other sites
so new work like this?

void *operator new(size_t size)
{
void *ptr = (void *)malloc(size);
};

Share this post


Link to post
Share on other sites
if you want a "debug" version of new, simply overload the new keyword to do your debug stuff. The default functionality will still happen. If you're doing a memory manager you'll want to look at placement new (basically you tell new where in memory to create your object: general usage is allocate a giant block of memory, keep it defragmented by moving stuff around and using placement new so that new objects are created in that pre-allocated block.

[google] "overload new C++" and "placement new"

-me

Share this post


Link to post
Share on other sites
Quote:
Original post by JasonL220
so new work like this?

void *operator new(size_t size)
{
void *ptr = (void *)malloc(size);
};


No! It's a built in keyword to the language. It doesn't use malloc. it also:

1) conctructs vtables
2) calls constructors

You cannot easily re-implement the functionality of new. just read my above post for what you want to do.

-me

Share this post


Link to post
Share on other sites
Quote:

No! It's a built in keyword to the language. It doesn't use malloc.


Not true.
It is a built in keyword, but it still uses malloc (at least VC.NET 2003's implementation does). You can check this by stepping into a call to new in the debugger.

Share this post


Link to post
Share on other sites
Quote:
Original post by perfectly_dark
Quote:

No! It's a built in keyword to the language. It doesn't use malloc.

Not true.
It is a built in keyword, but it still uses malloc (at least VC.NET 2003's implementation does). You can check this by stepping into a call to new in the debugger.

The new keyword != operator new.

The new keyword obtains storage by calling an allocation function, defined in section 3.7.3 to be the various forms of operator new(). How these functions obtain their storage is unspecified except that it shall be suitably aligned, at least as large as the requested size, not a null pointer for a size of 0, and that should the allocation fail it shall call the currently installed new_handler.

The new keyword is responsible for the construction of the type allocated as well, not operator new().

Share this post


Link to post
Share on other sites
You have to make a difference between the keyword new and the operator new, they're different things.
The keyword new allocates memory with an operator new call and then calls the constructor of the object.

Most compilers come with special debug-new implementations. You can pass parameters to operator new in the new-call, e.g. foo* p = new (std::nothrow) foo;

Writing an own memory manager is fairly advanced and you shouldn't start with overloading the new operator. Start by writing custom allocators (what an allocator is can be seen in the STL, where they're used rather than plain new-calls). They're much clearer then playing with the operator overloading.

Share this post


Link to post
Share on other sites
Quote:
Original post by Palidine
Quote:
Original post by JasonL220
so new work like this?

void *operator new(size_t size)
{
void *ptr = (void *)malloc(size);
};


No! It's a built in keyword to the language. It doesn't use malloc. it also:

1) conctructs vtables
2) calls constructors

You cannot easily re-implement the functionality of new. just read my above post for what you want to do.

-me


Note that there is a difference between "new" as a keyword, and "operator new".

Share this post


Link to post
Share on other sites
Unfortunately, some texts further the confusion by referring to the new keyword as the "new operator". So, you have the new operator and operator new. They are different.

As others have already explained, the "new operator" results in code that calls operator new() (which might use malloc), sets up inheritance details (such as the vtable pointer), and then calls the constructor. Or something like that.

Share this post


Link to post
Share on other sites
Quote:
Original post by JasonL220
so if i overide new it still calls the original new without any extra code

operator new() does memory allocation. That's what you override. The rest of the functionality of the new operator/keyword is unaffected.

Share this post


Link to post
Share on other sites
When you call new, does it null the memory ? (ie. fill it with 0's) ?

Cheers - Joe

Share this post


Link to post
Share on other sites
Just remember when your overwrite operator new, it will C++ will automaticaly call the default constructor of the object after it exit your 'new' operator function.

In regards to setting the memory to zero. I do the following in my code.

void *operator new(size_t size)
{
void *ptr = (void *)malloc(size);
memset( ptr, 0, size );
};

This is so all memeber variables of the class are set to NULL similar to java.

Share this post


Link to post
Share on other sites
doodo - that's actually a pretty good example of how not to replace operator new. Wait until you either run out of memory or perhaps make a zero byte request and then watch the SC++L fall over and die because your operator new returns a null pointer.

There are four things you can do with operator new. You can override it on a per-class basis (which will also apply for all derived classes unless they too overload operator new), you can overload it on a per-class basis (by providing versions with additional parameters), you can overload it globally (by providing a version with additional parameters) or you can replace it globally. The only restriction is that global placement new may not be replaced.

The normal forms of operator new all require that either a non-null pointer to dynamically allocated storage is returned or else an exception of type std::bad_alloc or of a type derived from std::bad_alloc is thrown. The nothrow forms of operator new all require that either a non-null pointer to dynamically allocated storage is returned or else a null pointer is returned.

The default behaviour of operator new is essentially equivalent to:
void * operator new(std::size_t size)
{
while (true)
{
void * memory = malloc(std::max< std::size_t >(size, 1));
if (memory)
{
return memory;
}
std::new_handler handler = std::set_new_handler(0);
std::set_new_handler(handler)
if (handler)
{
handler();
}
else
{
throw std::bad_alloc();
}
}
}
Σnigma

Share this post


Link to post
Share on other sites
Σnigma example is perfect and a near copy of mine besides a few different named variables and header/footer blocks wrapped around the allocated memory.

But what surprises me most is no one pointed the OP to Fluid Studios website and told him to grab the Memory Manager there that shows him all this. I actually found this when I was about 80% done mine but I was returning NULL (bad me, bad!). Anyways its a good reference to get you started.

@doodo: I think you need the above link too if that is in fact your entire new.

Best of luck JasonL220.

Share this post


Link to post
Share on other sites
Quote:
Original post by Enigma
doodo - that's actually a pretty good example of how not to replace operator new. Wait until you either run out of memory or perhaps make a zero byte request and then watch the SC++L fall over and die because your operator new returns a null pointer.Σnigma


Well enigma lets have a look at acouple of things before you start accusing me of some how breaking the convention.

Firstly, you cannot allocate a block of memory of size 0? why on earth would you do so? Even if you have a Empty class, the size of the class is exactly 1. The next thing is, you may assume that the instant address of the class i will be overwriting with 0 because of the mem set. Well unfortantly after the function exit's it will be calling the default constuctor that initalises all the member variables first, then after the default constructor is called it will call your default constructor that you have overwritten. This is a convention of C++, to ensure that all member variables are inialised before going into your constructor. To get to the point, the instant address of the object is not made during my 'function' it is created after it exits my function.

Share this post


Link to post
Share on other sites
Quote:
Original post by doodo
Quote:
Original post by Enigma
doodo - that's actually a pretty good example of how not to replace operator new. Wait until you either run out of memory or perhaps make a zero byte request and then watch the SC++L fall over and die because your operator new returns a null pointer.Σnigma


Well enigma lets have a look at acouple of things before you start accusing me of some how breaking the convention.

Firstly, you cannot allocate a block of memory of size 0?


Yes you can (kinda). The standard requires an implementation of new to return a unique pointer in such a case as well. While there are generally better alternatives, the address can then be used as a unique ID within the program. While I would recommend an alternative, it dosn't change the fact that this is /standard required behavior/, not just some ad hoc convention - and even if it were, null on out-of-memory would still be quite bad.

Quote:
Even if you have a Empty class, the size of the class is exactly 1.


You mean "greater than 0". The standard makes no requirement that it be 1 AFAIK.

I don't even understand what you're trying to talk about for most of your speil on the "instant address". If you're refering to the pointer returned by new, it's not part of the class - meaning it won't be nulled by the memset - and if it were, that'd still be nonstandard.

Share this post


Link to post
Share on other sites
@doodo:

operator new must not return a null pointer when zero bytes are requested:
Quote:
C++ Standard, Final Draft, Section 3.7.3.1 - [basic.stc.dynamic] - Allocation functions, Paragraph 2
The function shall return the address of the start of a block of storage whose length in bytes shall be at least as large as the requested size. There are no constraints on the contents of the allocated storage on return from the allocation function. The order, contiguity, and initial value of storage allocated by successive calls to an allocation function is unspecified. The pointer returned shall be suitably aligned so that it can be converted to a pointer of any complete object type and then used to access the object or array in the storage allocated (until the storage is explicitly deallocated by a call to a corresponding deallocation function). If the size of the space requested is zero, the value returned shall not be a null pointer value (4.10). The results of dereferencing a pointer returned as a request for zero size are undefined.30)
(emphasis mine).

I don't know why the C++ Standard requires zero-byte allocations to return a non-null pointer, but it does. Scott Meyers says:
Quote:
Effective C++ by Scott Meyers
In addition, the C++ standard requires that operator new returns a legitimate pointer even when 0 bytes are requested. (Beleive it or not, requiring this odd-sounding behaviour actually simplifies things elsewhere in the language.)

malloc, on the other hand, is not required to return a non-null pointer when zero bytes are requested:
Quote:
C99 Standard, Final Draft, Section 7.20.3, Paragraph 1:
The order and contiguity of storage allocated by successive calls to the calloc,
malloc, and realloc functions is unspecified. The pointer returned if the allocation succeeds is suitably aligned so that it may be assigned to a pointer to any type of object and then used to access such an object or an array of such objects in the space allocated (until the space is explicitly freed or reallocated). Each such allocation shall yield a pointer to an object disjoint from any other object. The pointer returned points to the start (lowest byte address) of the allocated space. If the space cannot be allocated, a null pointer is returned. If the size of the space requested is zero, the behavior is implementation-defined: either a null pointer is returned, or the behavior is as if the size were some nonzero value, except that the returned pointer shall not be used to access an object. The value of a pointer that refers to freed space is indeterminate.
(again, emphasis mine)

So if you implement operator new in terms of malloc without checking for zero-byte allocations your program fails to meet the requirements placed on it by the C++ standard and is not valid C++.

operator new may not return a null pointer if memory allocation fails:
Quote:
C++ Standard, Final Draft, Section 18.4.1.1 - [lib.new.delete.single] - Single object forms, paragraph 3 (in regards to void* operator new(std::size_t size) throw(bad_alloc);)
Required behavior: Return a pointer to dynamically allocated storage (3.7.3), or else throw a bad_alloc exception.

malloc on the otherhand, must return a null pointer if memory allocation fails. Check the quote from the C99 standard above. So if you implement operator new in terms of malloc without checking the return value of malloc and transforming null pointer returns into std::bad_alloc exceptions your program fails to meet the requirements placed on it by the C++ standard and is not valid C++.

Actually that last discussion is a little moot in your case, since you try and memset through the pointer. If malloc fails and returns a null pointer then your memset call has undefined behaviour. Try it yourself:
#include <cstddef>
#include <iostream>

// uncomment this to try your operator new
/*void * operator new(std::size_t size)
{
void * ptr = (void *)malloc(size);
memset(ptr, 0, size);
return ptr;
}*/


int main()
{
try
{
void * p = operator new(std::size_t(-1));
}
catch (std::bad_alloc & exception)
{
std::cerr << "Failed allocation\n";
}
}




I'm not "assume that the instant address of the class i will be overwriting with 0 because of the mem set" at all (assuming I'm deciphering that correctly). The contents of the memory allocated is not an issue - I'm well aware of what happens in a call to the new operator. The issue is what happens when memory isn't allocated.

Σnigma

Share this post


Link to post
Share on other sites
Would overloading new as doodo did enable me to catch memory allocation failure by comparing the pointer returned by new with NULL instead of using exception on every bit of code that uses "new" (which I find to be a pain in fact) ? And countrary to just using malloc, I could still take advantage of new calling the constructor.

Is not conforming to C++ standards a real big issue and why ? (Provided all team members are aware of how the "custom new" behaves, etc.)

(there no irony here *at all*, I would just like to know that from experienced ppl here, thanks :) )

Share this post


Link to post
Share on other sites
Quote:
Original post by janta
Would overloading new as doodo did enable me to catch memory allocation failure by comparing the pointer returned by new with NULL instead of using exception on every bit of code that uses "new" (which I find to be a pain in fact) ? And countrary to just using malloc, I could still take advantage of new calling the constructor.

No. Although the standard doesn't appear to be entirely clear on the issue it appears that the standard new operator is allowed to always try to call the constructor unless an exception is thrown. So if you replace operator new with a version which returns null then the new operator may attempt to construct your object at an invalid location in memory, invoking undefined behaviour and probably crashing your program. Even if it doesn't, the standard library relies on allocations either allocating memory or throwing exceptions and will do things roughly equivalent to:
type * thing = new type;
*thing = other_thing;

If new type returns a null pointer then BOOM - welcome to undefined-behaviour land.

That said, you shouldn't be needing to explicitly "use exceptions" on evey bit of code that uses new, but which I assume you mean:
try
{
pointer = new thing(/* arguments */);
}
catch (std::bad_alloc & exception)
{
// do something
}

Instead, you should allow the exception to propogate to a level at which the exception can be handed correctly.

If you really need to allocate memory without an exception being thrown then use the nothrow form of new:
thing * t = new (std::nothrow) thing(/* arguments */);
if (!t)
{
// do something
}

This is guaranteed not to go BOOM.

Quote:
Is not conforming to C++ standards a real big issue and why ? (Provided all team members are aware of how the "custom new" behaves, etc.)

If you don't conform to the standard then you can't be sure how your code will port to a different platform, compiler or even to the same compiler with a new patch/service pack. In the case of the new operator you might also lose the ability to reliably use the standard library. Failure to conform to the standard rarely gains you anything, although there are exceptions (but you have to be pretty experienced to know what said exceptions are and how to use them). In general stick to the standard unless you have a very good argument against. "it makes things easier for me" and "well, it's worked every time I've tried it so far" are not very good arguments.

Σnigma

Share this post


Link to post
Share on other sites
Quote:
No. Although the standard doesn't appear to be entirely clear on the issue it appears that the standard new operator is allowed to always try to call the constructor unless an exception is thrown. So if you replace operator new with a version which returns null then the new operator may attempt to construct your object at an invalid location in memory, invoking undefined behaviour and probably crashing your program.

A pretty obvious point I feel ashamed not having thought of ;)

Share this post


Link to post
Share on other sites
It's easy enough to conform to the standard and you really want too. Like I said in my other post, check out the memory manager from Fludio Studios. It does an excellent job of implementing new and will show you how to properly do it. It's not that many lines of code (20-30).

As for requesting a size 0 with new, I check for it and make it 1 (not sure if that's what the standard requires but it's what I do).

@Jason: I really recommend implementing it correctly.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement