• Advertisement
Sign in to follow this  

new() question

This topic is 4142 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

In Visual C++ 2005, if I were to make an unlimited memory loop like this: while(1) { new int[1000]; } I would expect to eventually run out of memory. However, the program seems to be able to recycle memory, or reuse it somehow. Is it going into virtual memory? Or does VC++ 2005 have some form of rudimentary garbage collection?

Share this post


Link to post
Share on other sites
Advertisement
good question :)

i dont know the answer, so i ran your program in VC 2003
then viewed the program in the task manager.

The memory usage didn't increase a whole lot (in the Processes tab), but the page file (in the performance tab) size increased steadily until the application finished. So i guess the memory being allocated is being swapped out to virtual memory.

Share this post


Link to post
Share on other sites
The process will quickly eat up available RAM, and then move to paging. Once it starts doing that, things will really start to slow down. It would take quite a bit of time to exhaust a decently large page file at that rate.

Share this post


Link to post
Share on other sites
Nope, no garbage collection, unless you use .NET and gcnew.

This program will run out of memory and crash, but at that rate it will take a while. Try adding a few more zeros and see it more rapidly come to a screeching halt.

Share this post


Link to post
Share on other sites
As that code has no external side effects, it's not guaranteed to run at all.

To find out exactly what your compiler decided to do check the assembly produced.

Share this post


Link to post
Share on other sites
isn't the allocation a side effect because its allocating a dynamic object?
Here is the asm produced:


PUBLIC _main
EXTRN ??2@YAPAXI@Z:NEAR ; operator new
; COMDAT _main
_TEXT SEGMENT
_main PROC NEAR ; COMDAT

; 4 : {

$L272:

; 5 :
; 6 : while(1)
; 7 : { new int[1000]; }

push 4000 ; 00000fa0H
call ??2@YAPAXI@Z ; operator new
add esp, 4
jmp SHORT $L272
_main ENDP
_TEXT ENDS
END


Share this post


Link to post
Share on other sites
Quote:
Original post by Nitage
As that code has no external side effects, it's not guaranteed to run at all.

To find out exactly what your compiler decided to do check the assembly produced.


Isn't new() an external side effect?

Share this post


Link to post
Share on other sites
new doesn't necessarily throw exceptions, and if that's the case the program could run forever.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
new implementation is OS dependant. Just calling new without touching the memory could very well just mark space (in the virtual address space) as reserved. The actual "allocation" as we normally know it happens on first read or write (having physical ram targeted for the specific allocation).

What is probably happening is virtual address space is consumed with each iteration. Eventually, new throws an exception, which is unhandled, and the process exits. Alternatively, the process may exit when virtual address space is exhausted (depending on the implementation of new).


Share this post


Link to post
Share on other sites
Quote:

new doesn't necessarily throw exceptions, and if that's the case the program could run forever.

A standards-compliant implementation of new always throws if allocation fails. The nothrow overload of new (which was not used here) will suppress the bad_alloc exception and instead return null.

Share this post


Link to post
Share on other sites
Quote:
Original post by jpetrie
Quote:

new doesn't necessarily throw exceptions, and if that's the case the program could run forever.

A standards-compliant implementation of new always throws if allocation fails. The nothrow overload of new (which was not used here) will suppress the bad_alloc exception and instead return null.


While it's true that a standards-compliant implementation of ::operator new always throws if there is not replacement std::new_handler and the allocation fails, there is no guarantee that the allocation will fail even if new is unable to allocate memory, so there is no guarantee that new will throw std::bad_alloc if it is unable to allocate memory, and that behaviour is implementation-specific.

For example, most Linux and BSD Unix distributions use a strategy called "overallocation" in which brk() will not indicate failure even when it has exhausted all available virtual memory. You will never see new throw bad_alloc on such systems unless you modify tunable kernel parameters to disable such functionality. What you will see is either the OOM (out-of-memory) killer terminate your process, or the system itself will appear to lock up and you will exercise the power switch to recover, depending on the version of the kernel.

So, in short, the AP was correct. The throwing behaviour of new is in fact OS dependant. then again, it's not completely predictable anyway since you can implement your own std::new_handler to avoid throwing.

Share this post


Link to post
Share on other sites
Quote:
Original post by Bregma
For example, most Linux and BSD Unix distributions use a strategy called "overallocation" in which brk() will not indicate failure even when it has exhausted all available virtual memory. You will never see new throw bad_alloc on such systems unless you modify tunable kernel parameters to disable such functionality. What you will see is either the OOM (out-of-memory) killer terminate your process, or the system itself will appear to lock up and you will exercise the power switch to recover, depending on the version of the kernel.


And the benefit of doing things that way is what, exactly? o_O (Also, how would the "OOM killer" detect the condition?)

Share this post


Link to post
Share on other sites
Quote:
Original post by Ozymandias42
Quote:
Original post by Nitage
As that code has no external side effects, it's not guaranteed to run at all.

To find out exactly what your compiler decided to do check the assembly produced.


Isn't new() an external side effect?


Nope. From the C++ standard:

Quote:

conforming implementations are required to emulate (only) the observable behavior of the abstract machine as explained below.


Quote:

The observable behavior of the abstract machine is its sequence of reads and writes to volatile data and calls to library I/O functions

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Quote:
Original post by Nitage
Quote:
Original post by Ozymandias42
Isn't new() an external side effect?


Nope. From the C++ standard:

Quote:

conforming implementations are required to emulate (only) the observable behavior of the abstract machine as explained below.


Quote:

The observable behavior of the abstract machine is its sequence of reads and writes to volatile data and calls to library I/O functions

But a call to new is bound to write some volatile data in the kernel - It can be observed by other processes as rising memory usage.

Share this post


Link to post
Share on other sites
new int(1000) - thats 4000 bytes of memory per allocation - to fill up 256 MB of ram would take 67000 allocations - Then there's paging which will make allocations slower. Check the process monitor(task manager/'ps') to see if your program is hogging memory?

Share this post


Link to post
Share on other sites
The C++ standard doesn't specify how new works. It certainly isn't bound to write to volatile data.

What exactly does the code do when you run it?

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Quote:
Original post by Nitage
The C++ standard doesn't specify how new works. It certainly isn't bound to write to volatile data.
But it doesn't specify that writing to volatile memory is illegal either, does it? So your original claim "As that code has no external side effects, ..." may or may not be true, depending on the implementation of new. Knowing that he compiled with Visual C++ 2005 and didn't claim anything about having overloaded the new operator, the new he's using indeed does have external side effects that can be observed by checking the amount of free memory available. So for that particular implementation of new, it would be against standard to optimize away the new.

Share this post


Link to post
Share on other sites
Quote:
Original post by Zahlman
Quote:
Original post by Bregma
For example, most Linux and BSD Unix distributions use a strategy called "overallocation" in which brk() will not indicate failure even when it has exhausted all available virtual memory.

And the benefit of doing things that way is what, exactly? o_O (Also, how would the "OOM killer" detect the condition?)


The benefit is one of speed. What happens is that the page is not physically allocated until it is written to, so it's possible to allocate a slot in the page allocation table and have that allocation succeed, but when an attempt is made to write to the page, an actual page frame needs to be allocated and at that point an error will occur. That means either the kernel will detect that there is a system-wide out-of-memory situation and the OOM killer will come alive to free up some page frames, or the app will get an invalid-address or bus-error signal and terminate. It depends on the OS.

The OOM killer can detect this situation because it's a part of the kernel, and the kernel knows when core memory and swap memory is filled, and it knows who is filling it. When it detects the situation that virtual memory is exhausted (core and page are completely allocated), it will kill the process with the highest OOM score, which by default is the process with the largest page allocation table. Most of the time, that's a program with a problem, so most of the time it's the right thing to do.

What it comes down to is that on some systems, the new will succeed but any attempt to use the memory allocated may fail. Alternatively, the new may succeed even though the underlying physical memory have not been allocated, and then some other process is killed and the page frame request is fulfilled via the OOM killer whacking some pig and the happy app continues on its merry way, unaware of what's going on under the hood.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement