• Advertisement
Sign in to follow this  

What happens with 32bit overflow?

This topic is 770 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I wonder what happens if a 32bit process runs out if its memory limitation running on a 64bit system, will just a heap allocation be refused thus making program terminate exeptionally?

Share this post


Link to post
Share on other sites
Advertisement

"32 bit process" means "I don't need more address space than 32 bits", even if I might run at a 64 bit system.

Eg pointers in registers and memory  are then all 32 bit, it's simply not possible to point to address at or above 2**32.

 

So indeed, it's "no more addressable memory, game over, have a nice day".

Share this post


Link to post
Share on other sites
will just a heap allocation be refused thus making program terminate exeptionally?

 

 
If there is no available memory, a memory allocation request will fail.
 
This does not mean the program will immediately terminate, necessarily, although in practical terms it does. Most programs can't do much in an out-of-memory scenario and so will either terminate themselves or ignore the failure and allow the eventual subsequent issues to terminate the process. It is possible in to handle and recover from a failed memory allocation attempt and continue running, however.

Share this post


Link to post
Share on other sites


It is possible in to handle and recover from a failed memory allocation attempt and continue running, however

You see mobile games handling this much more often, as memory is often quite constrained on mobile devices, and the OS tends to be much more aggressive about closing applications that use too much memory.

 

Although it is worth noting that the allocation call itself rarely actually fails on mobile. Instead, both Android and iOS have high-level events which will (hopefully) be delivered to your applications before the system runs out of memory, and if you continue to allocate more memory in such a situation, the OS will shortly terminate your process.

Share this post


Link to post
Share on other sites

It's not a simple question actually.
 

Virtual address is not the same as virtual memory.

 

32-bit processes cannot address more than 2^32 bytes of memory. However, they can access more than that considering they only work on sections of it at a time and remap different chunks of virtual memory into virtual addresses as they need.

With this trick, on 32-bit OSes, up to 64GB of memory can be accessed (assuming PAE is enabled). On 64-bit OSes I don't know if the 64GB limit still applies, or if they can use more.

Here's an example on how to do that: https://blogs.msdn.microsoft.com/oldnewthing/20040810-00/?p=38203

 

You will notice that you need OS specific calls to do this. Regular malloc won't do.

 

Going back to the original question, what will happen? Well it depends of how the program is coded:

  1. Apps using malloc that run out of virtual addressing ranges, will get a null pointer. If the app checks for this condition, it can successfully try to recover (i.e. start to free memory and try again, show an error and quit, silently quit, etc). If the app doesn't check for this condition, it will continue to execute normally until the null pointer is used which will lead to a crash. Usually malloc'ed memory is used immediately afterwards, so the crash will happen close to the malloc call; but who knows how much the crash could be delayed if the null pointer isn't accessed soon. It's also likely many subsequent malloc calls will fail, which means a crash is imminent unless a lot of memory suddenly became available again.
  2. Apps using C++'s new operator will throw an exception when it runs out of memory; so it will crash immediately. However the app can catch this exception and try to recover, like in the malloc case. Poorly written C++ code may catch all exceptions and not notice it's catching an out of memory exception; in which case they'll continue to execute except now they have an uninitialized pointer (which worse than a null pointer). Uninitialized pointers normally crash immediately when they're used, or may corrupt the process' memory and continue to run until it's so broken it can't work anymore or crashes (think Gameboy's Pokemon glitches)
  3. Apps using other languages (i.e. Python, Ruby, Java) depend on how their interpreter manages out of memory conditions, but most of them handle it by just crashing or raising an exception (also depends on whether the interpreter couldn't honour the amount of memory the executed code is requesting; or if the interpreter itself ran out of memory while inside of its internal routines).
  4. Apps manually managing memory (the CreateFileMapping method in the blog) are advanced enough to assume they're handling this condition. When out of memory, CreateFileMapping and/or MapViewOfFile will fail. The rest is as the malloc case.

Edit: What I wrote is Windows/Linux/OS X specific. As described above by swiftcoder, iOS and Android are much more strict about memory (they just immediately terminate a process that starts consuming too much memory)

Edited by Matias Goldberg

Share this post


Link to post
Share on other sites

  • Apps using C++'s new operator will throw an exception when it runs out of memory; so it will crash immediately. However the app can catch this exception and try to recover, like in the malloc case. Poorly written C++ code may catch all exceptions and not notice it's catching an out of memory exception; in which case they'll continue to execute except now they have an uninitialized pointer (which worse than a null pointer). Uninitialized pointers normally crash immediately when they're used, or may corrupt the process' memory and continue to run until it's so broken it can't work anymore or crashes (think Gameboy's Pokemon glitches)

Basically true, although you can specifically override that behavior with std::nothrow. However while it should be mentioned for completeness sake I have never actually seen it in the wild.

Share this post


Link to post
Share on other sites

Basically true, although you can specifically override that behavior with std::nothrow. However while it should be mentioned for completeness sake I have never actually seen it in the wild.

It was pretty common on Android for a while, because the C++ compiler and runtime didn't support exceptions for the longest time. These days they do, but a lot of code is still compiled with exceptions disabled.

Although, at least half the code I have seen incorrectly assumes that disabling exceptions is the same as std::nothrow, in which case, allocation failures will be silent and deadly.

Share this post


Link to post
Share on other sites

It's worth noting that there's some extra Windows specific detail here:

 

- On a 32-bit version of Windows, you don't get 4GB of address space. You get 2GB. The other 2GB is used by Windows (for cache, drivers, etc).

- On a 64-bit version of Windows, a 32-bit program also gets 2GB of address space by default, but you can link your program with /LARGEADDRESSAWARE and get 4GB.

- If you compile your program as 64-bit then you get lots more address space - up to 128TB (48 bit addresses, with half of the address space used by the OS).

 

Note that for simplicity I'm ignoring the /3GB and /USERVA boot flags, as it's not generally something you can control on the PC your software will be installed on.

Edited by Adam_42

Share this post


Link to post
Share on other sites

Note that for simplicity I'm ignoring the /3GB and /USERVA boot flags, as it's not generally something you can control on the PC your software will be installed on.


Yes and no. To the best of my knowledge /3GB is initially enabled on all Windows systems more modern than XP. Since XP has been officially deprecated for a while, that should not have been a dealbreaker. Enabling support for /3GB does not hurt you even when the operating system does not support it though (you just won't get the extra memory). I think Age of Conan added /LARGEADDRESSAWARE support pretty soon after it came out because it really helped the performance and x64 executables weren't the rule around that time.

Share this post


Link to post
Share on other sites
Unless of course your legacy codebase or dependencies are not playing nice with 64 bits. Straight-forward problems like trying to store a 64bit pointer in an integer will at least produce reasonable warnings but I have seen some truly horrible things which just blow up in interesting ways...

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement