Sign in to follow this  
Krohm

'new' issue

Recommended Posts

From the beginning, I always used new pretty much like malloc with some extra machinery. Let me say that my 'beginning' means many years ago. At the time, exceptions were still somewhat clunky, and although they worked in theory, I have to say their implementation were terrible. So, new at the time just returned NULL on failure, which was fine to me at the time, although we all know how much the subsequent pointer checking is.
float *vec = new float[nComps * nvtx];
if(!vec) return false; // how boring
In 2009 I started to think that I could transition to the new exception-based new (which happens to be the standard even for MS libs now): after all, this is an excellent application for exceptions, which in the meanwhile have matured alot. So I started to drop the pointer checking. Today, I read: A call to new may return NULL if the platform has run out of memory, so test the return value before dereferencing it.. Uh-oh. So, is the throwing new standard or not? Is he referring non-PC platforms? I admit that I have no multiplatform experience at all and minimal multi-OS experience.

Share this post


Link to post
Share on other sites
Yes, new should throw exception when it can not allocate requested memory
If you don't want for it to throw exception, then use nonthrowing new:
#include <new>

...
float* p = new (std::nothrow) float [count];

Share this post


Link to post
Share on other sites
Yes, a throwing 'new' is standard C++ '98, though it took some years before Microsoft released a new C++ compiler after their, uhm, "groundbreaking" MSVC++ 6.0.

edit: ninja'd++.

Share this post


Link to post
Share on other sites
Thank you both for your fast reply,
however, I am sort of unsure I am communicating correctly my concerns.

The non-throw'ing null is well known to me. It is not something I want to see around in my code. I actually don't want new to NOT throw exceptions, I want to migrate to the throwing null, which implies always valid pointers out of it.

And, yes, I know the throwing NULL is standard since years, as well as my refusal to use exceptions in most situations is probably not justified those days.

Yet the article I linked comes from a source I personally find trustable and if they write that new can still return NULL I am left wondering what they're talking about.

I have tens of thousands of line of code using NULL-new accumulated for reuse over the years. Although inelegant by my current likings, they will work with throwing NULL. It is forward compatible.
By contrast, the throwing null approach is not back compatible. Although the possibility I port to some esoteric or closed platform is minimal, I still like to believe it is not zero.
Now, in the admittedly low chance this happens, if the platform doesn't support throwing new, all this code will turn unsafe.
Seeying it in perspective, knowing that MSVC will play nice and that "it is standard since" ten years ago, leaves me at the start - I don't think this example was random.

Share this post


Link to post
Share on other sites
Then, what exactly is your question if it's not

Quote:
So, is the throwing new standard or not?


because a throwing new is standard. If new doesn't throw an exception upon out-of-memory, then that behaviour is compiler specific and non-standard. In that case, you must check the manual of the compiler you are going to use. Imho, it isn't worth the effort to take into account all possible compiler specifics until you use that compiler, and until you write a library. Fr instance, if you look at boost, you will see that they coded their library to be compatible to many compilers, and that lead to total code uglyness in some parts of their files.

If you ask me, yagni. The less you have to write, the less error prone is your code.

Out of curiosity: Does the guy say that "in C++, new can return NULL", or does he say "under the compiler foo, new can return NULL"? Because the former is false, and would have to be rephrased to the latter.

Share this post


Link to post
Share on other sites
Since the article mentions programming for the Wii it's probably that they've turned off exceptions since they have a non-trivial amount of overhead. Given this it's probably that the compiler switches to no-throw new everywhere, or they're using a custom allocator which returns NULL instead of throwing an exception.

Share this post


Link to post
Share on other sites
Quote:
Original post by Krohm

I think I will just keep NULL-new convention for another couple of years.


Stick with COBOL instead.

Quote:
I admit that I have no multiplatform experience at all and minimal multi-OS experience.


For you - new is the standard and only way. It allows to progress beyond C and 15 years old concepts as well as use modern libraries (written in last 10 years).

Just because some project that you did not develop, that you have never worked with, that you don't plan to work on and that you will never encounter chose to not use exceptions (they serve no real use on Wii - if it breaks, console breaks), is not even remotely a legitimate reason to avoid something.

It gets even trickier. On systems with virtual memory, even if new succeeds, you are not guaranteed to have actually claimed the memory. Since large blocks do not get committed, it is perfectly possible to have new succeed, yet fail when accessing the memory on a new page when it is actually committed. See overcommit.

So if you want to be sure, you need to write custom allocator, that will write all allocated bytes, and then either return the pointer, or crash.

But since few have heard about this problem, it just goes to show how irrelevant allocation failures have become on every desktop OS.

And if you are working on system with limited memory, the whole process becomes completely different anyway, so null will be the least of your worries.

Share this post


Link to post
Share on other sites
Just for reference, from TC++PL §6.2.6.2 "Memory Exhaustion":
Quote:

What happens when new can find no store to allocate? By default, the allocator throws a bad_alloc exception [...]

I could not find anything in the standard after a few minutes of searching. Anyone?

Share this post


Link to post
Share on other sites

float *vec = new float[nComps * nvtx];


If you do something like this, your code probably isn't exception safe in the first place - if new indeed did throw somewhere, you'd leak memory from all the other naked pointers that you might have around. (Or you'll need to handle the exception very close to the source, which would be quite tedious too.)


std::vector<float> vec(nComps * nvtx);


Leak-proof now.

Share this post


Link to post
Share on other sites
Quote:
Original post by visitor

float *vec = new float[nComps * nvtx];


If you do something like this, your code probably isn't exception safe in the first place - if new indeed did throw somewhere, you'd leak memory from all the other naked pointers that you might have around. (Or you'll need to handle the exception very close to the source, which would be quite tedious too.)


std::vector<float> vec(nComps * nvtx);


Leak-proof now.


Or:

auto_ptr<float> vec = new float[nComps * nvtx]
.

shared_array<float> vec = new float[nComps * nvtx]


But then it's not for beginners anymore, I guess.

edit: according to rip-off's. mea culpa.

Share this post


Link to post
Share on other sites
Quote:
Original post by phresnel
auto_ptr<float> vec = new float[nComps * nvtx]
.

But then it's not for beginners anymore, I guess.

You can't use auto_ptr for an array, it will free using delete, not delete[].

Share this post


Link to post
Share on other sites
Quote:
Original post by DevFred
Just for reference, from TC++PL §6.2.6.2 "Memory Exhaustion":
Quote:

What happens when new can find no store to allocate? By default, the allocator throws a bad_alloc exception [...]

I could not find anything in the standard after a few minutes of searching. Anyone?


What are you asking exactly?

Share this post


Link to post
Share on other sites
Quote:
Original post by DevFred
I could not find anything in the standard after a few minutes of searching. Anyone?


This is the best I could find from a few minutes of looking through my copy of the standard.

The standard definitions of operator new (3.7.3.2):
Quote:

void* operator new(std::size_t) throw(std::bad_alloc);
void* operator new[](std::size_t) throw(std::bad_alloc);


Followed by 3.7.3.1.3:
Quote:

If an allocation function declared with an empty exception-specification (15.4), throw(), fails to allocate storage, it shall return a null pointer. Any other allocation function that fails to allocate storage shall only indicate failure by throwing an exception of class std::bad_alloc (18.4.2.1) or a class derived from std::bad_alloc.

Share this post


Link to post
Share on other sites
Quote:
Original post by Krohm
Yet the article I linked comes from a source I personally find trustable and if they write that new can still return NULL I am left wondering what they're talking about.
As credible as the source might seem, the statement in question is nevertheless false. One would need to quantify the statement with specific old compilers at the very least.

If you'd like to help keep up the credibility of that source, then by all means please contact the site or article author and let them know of the mistake.

Share this post


Link to post
Share on other sites
Quote:
Original post by iMalcAs credible as the source might seem, the statement in question is nevertheless false. One would need to quantify the statement with specific old compilers at the very least.

If you'd like to help keep up the credibility of that source, then by all means please contact the site or article author and let them know of the mistake.


The source is just fine, read the article, but more importantly, the context it was written in.

The quote isn't about some weird C++ standard issues, or some obscure platforms. The paragraph comes through as if developer seemed surprised that on embedded hardware it is necessary to check for memory allocation failures (given a console projects, exceptions were probably disabled, or aren't supported).


The article is written from a junior developer's perspective (as stated in article).

Other advice states to validate parameters and expect that sometimes your function will be called with unexpected parameters. Another paragraph says to learn bit flags. And another that requirements change.

The surprise over memory allocation failures is common. Memory allocation is an obscure and arcane topic to majority of new developers, and essentially all that have been raised on cups of Java. Majority of software developers today has no concept of resource cost (memory, disk, CPU - "what, I tested with 5 files, it works fine, what do you mean O(n^5)"), and many (especially IT-centric) practices strictly forbid thinking about cost, which results in gigabyte behemoths where no resources could have been claimed at all.

This isn't limited to small projects either. I ran into problems upgrading eclipse, when something looked fishy after taking 30 minutes with no apparent progress. It turned out, that upgrader was creating hundreds of thousands (100,000+) directories with a handful of files in them (millions of 100 byte file system entries). It was simple design (reflection or something mapped to file system), but it's the absolute worst-case scenario for NTFS. After moving temp location to RAM drive, the whole process took only a few seconds.

Share this post


Link to post
Share on other sites
Quote:
Original post by Antheus
The quote isn't about some weird C++ standard issues, or some obscure platforms. The paragraph comes through as if developer seemed surprised that on embedded hardware it is necessary to check for memory allocation failures (given a console projects, exceptions were probably disabled, or aren't supported).
Thank you, I think you have got my point of view perfectly. I currently always check'em and I wanted to know if I could drop out the check. Turns out this is not possible on at least one major platform.

I don't think I need to request a fix. To me, it was very clear he was referring to a specific implementation, you know how those things usually work.
I admit my initial wording ('standard') was overlooked. I didn't really mean that but rather 'commodity', de jure standards don't get me excited until they prove to be estabilished (yes, I understand your opinions about the C++ standard and such - in 2009 one would expect them to be universally accepted). I apologize for the confusion.

Share this post


Link to post
Share on other sites
Quote:
Original post by loufoque
Quote:
Original post by DevFred
I could not find anything in the standard after a few minutes of searching. Anyone?

What are you asking exactly?

Since 1998, there is a document that describes C++ in every detail. If new throws an exception, that should be noted somewhere in that document.

Share this post


Link to post
Share on other sites
My bad, I just thought you quoted the standard while you quoted a book...

It's in 3.7.3.1/3.

C++ with new that doesn't throw, or even C++ without exceptions, is simply not C++ anymore.

Share this post


Link to post
Share on other sites
Quote:
Original post by Krohm
Thank you, I think you have got my point of view perfectly. I currently always check'em and I wanted to know if I could drop out the check. Turns out this is not possible on at least one major platform.


That isn't what I said.

C++ standard defines two implementations of new operator.
1) Common new, which fails by throwing std::bad_alloc
2) Placement std::nothrow new, which doesn't throw, but returns null on failure

Simply put: If you intend to avoid exceptions, you need to use the nothrow version of operator new. Or, the following:
float *vec = new float[nComps * nvtx];
if(!vec) return false; // how boring
will always succeed, since new will either succeed, throw an exception, or cause undefined behavior if exceptions are disabled. Proper version would be:
float *vec = new (std::nothrow) float[nComps * nvtx];
if(!vec) return false; // how boring
or better yet:
std::vector<float> vec(nComps*nvtx);
The later one implies exceptions as well.

This gives you proper behavior on standard-compliant compilers. For others, there are no guarantees, not even with respect to null.

But usually, when one needs to worry so badly about memory, different strategies are preferred anyway. Such as assigning a fixed amount of memory to each system, handed out from pre-allocated storage. This way, all the OS allocation requests are performed up-front, everything else is application controlled.

Share this post


Link to post
Share on other sites
Quote:
Original post by AntheusThis gives you proper behavior on standard-compliant compilers. For others, there are no guarantees, not even with respect to null.
Your point is taken; I still find unreasonable that a non-conformant new would return non-NULL on failure as this was how it worked in the beginning and how it worked with malloc. This is also the behaviour the linked article is referring to.
Sure, there could be an implementation not honoring the exception rule and still not honoring oldskool NULL semantics just because the spec doesn't forbid this behaviour.
The question is: does anyone knows of an implementation behaving that way?

As said, de jure standards take me only up to a certain point, and while concluding that failing-non-conformant new returns NULL is not supported by the specification, concluding that it returns non-null is just the same but clashing with habits in addition to the spec; furthermore i see little to no low-level reason to do so even on the most limited CPU I know about and would imply that even malloc couldn't be implemented on it (or maybe that the implementation shipped with a quirk so colossal no word can describe). It seems reasonable that at least the basic C semantics will be honored by all commercially sound systems & some extra ones.

Yes. Non-conformant new sucks. We can freely say some environments truly suck (and this isn't even the worse I am used to). Said so, everything will carry on as it was.

Quoting myself:
Quote:
The non-throw'ing null is well known to me. It is not something I want to see around in my code. I actually don't want new to NOT throw exceptions, I want to migrate to the throwing null, which implies always valid pointers out of it.
... or nothing at all I may add. Whatever I will leak a few KBs of data (or not) upon throwing is the least of my concerns.
Since I first heard that new would have got throwing semantics I knew it would have been fatal to a large part of the codebase. I actually find this positive as the implication of a badalloc is always something gone very wrong in some previous stage. It is very borderline scenario.

It is quite a while I'm considering replacing all new with new(nothrow). The only thing I can say after this discussion is that the current approach will work more or less as expected in all situations I'm aware of; the throwing null won't in at least one realistic context.
I suppose I will consider again in 2014 or something like it. Maybe all implementations will be conformant by the time. :P

Thanks to all members for their contribution. Self-replicating coffee follows.

)
(
)
,.----------.
((| |
.--\ /--.
'._ '========' _.'
jgs `""""""""""""`

EOT

Share this post


Link to post
Share on other sites
Quote:
Your point is taken; I still find unreasonable that a non-conformant new would return non-NULL on failure as this was how it worked in the beginning

What beginning?
It always has been that way since the first C++ standard.

Quote:
and how it worked with malloc.

That's irrelevant. C++ is totally different from C.

If you want C-like semantics, why don't you use C in the first place?

Quote:
Whatever I will leak a few KBs of data (or not) upon throwing is the least of my concerns.

The most important property of decent C++ programs is exception safety, which guarantees no resource is ever leaked even in the face of exceptions.
This is usually achieved by using RAII, which itself requires exceptions.

And unlike what you think, all implementations are conformant about this. Yes, even console ones.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this