Does more instructions in a program use more RAM than programs with few instructions?

This topic is 1418 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

Recommended Posts

Everytime a program runs, the entire code of the program gets loaded into main memory. So can I come into a conclusion and say that it is true a program with more instructions use more ram than programs with few instructions?

Is that why games requires a certain amount of RAM because they contain a lot of instructions and also a lot of instructions that needs to be processed by the CPU?

Share on other sites

Things like models, textures, sound files etc are orders of magnitude larger than exes.

I worked with sound files in games for my 2D games. And yes those sound files do have large file size. For reasons I do not know.

Why is models and textures orders of magnitude large? Why does it used a lot of RAM?

Share on other sites

Things like models, textures, sound files etc are orders of magnitude larger than exes.

I worked with sound files in games for my 2D games. And yes those sound files do have large file size. For reasons I do not know.

Why is models and textures orders of magnitude large? Why does it used a lot of RAM?

Well, think about a typical instruction in machine code. Varies depending on the hardware but you'll fit an entire instruction including its parameters in, say, four bytes. Four bytes is also how much it would take to store one pixel in an uncompressed RGBA texture. So a 256 x 256 texture is using the same amount of memory as 65,535 of my imaginary instructions.

65k instructions would cover a lot of program logic. A 256 x 256 texture is kind of titchy.

Share on other sites

Everytime a program runs, the entire code of the program gets loaded into main memory. So can I come into a conclusion and say that it is true a program with more instructions use more ram than programs with few instructions?
It is not always the case that the entire program code is loaded into memory. In the cases where that is actually true, then yes more code uses more memory. This can actually become a serious problem on the older consoles (PS3/Xbox360 and similar) as well as some mobile devices, because you are often running very close to memory limits on those devices. However on PC operating systems the executable will typically be paged into active memory as needed, plus there's plenty of memory to begin with most of the time.

Share on other sites

Everytime a program runs, the entire code of the program gets loaded into main memory. So can I come into a conclusion and say that it is true a program with more instructions use more ram than programs with few instructions?

Yes. Some games/engines have problems running on constrained devices like the Wii because they can't load the executable (and needed resources) into memory (I've seen engines where just the executable by itself outstrips the Wii's meager memory constraints).

Generally, as others have said, the game's resources will swamp the size of the executable, though. Typical ballpark game EXEs are around 5-20MB while typical game assets (just the portion needed to display a particular scene/level) are generally 100s of MB, and 1,000s of MB on some of the newer titles.

The size of _one_ resource vs the executable isn't an issue. The executable may well be quite a bit larger than most resources, especially if properly compressed. A single scene is likely to require hundreds or thousands of resources loaded up at once, however, and they all add up to way more space than an executable. You only need one set of rendering/physics/game logic but you need lots of different data to feed that logic to get a game.

Share on other sites

Well, just do the math, a 32x32 24 bits texture only use 2208 bytes, while a 4096x4096 texture take about 50 mb, so, it scale up fast, not to mention video cutscene. Like others said, it's not the .exe that take this space, it's the amount of allocated memory to hold those assets, for example, using new and malloc in c++.

Share on other sites

Here's an example from Elder Scrolls Online.

'data' - 24.1 Gb

audio files - 5.02 Gb

video - 92.6 Mb

game.exe - 29.1 Mb

Don't forget any DLLs shipped with the game. Those would be loaded into memory as well, and count as part of your "exe" statistics.

Share on other sites

For a recent PC game we pushed up to about 2.5 GB of data loaded at any time. (Windows allows a 32-bit program a maximum of 3 GB address space). On certain old min-spec machines this meant some page file swapping, but for most gamers that will fit in main memory.

That 2.5 GB of data was almost entirely textures, models, and audio. The executable itself along with all the associated libraries required less than 45MB, or just under 2% of the total size.

Adding a few bytes to the executable is generally not significant for a major game title.

Is that why games requires a certain amount of RAM because they contain a lot of instructions and also a lot of instructions that needs to be processed by the CPU?

No.

You can write (and people have written) very small programs that process many terabytes of data. The size of the program does not directly correspond to the data needs.

If you want some fun examples, look back to the old PC demoscene programmers. Most were written in hand-crafted assembly while carefully watching their binary size. Many of them wrote procedural graphics. Quite a few of the later programs relied on D3D for their graphics engines. A small number of them designed systems that extended to the full amount of available memory on the system, capping out at the system's addressable limits. Re-running some of those old demoscene programs that are just a few kilobytes in size and end up processing gigabytes of graphics, well, they are just amazing.  They are basically screensavers, but still amazing.

Or for a non-games industry... Some quick searching shows Walmart averages over 1 million point of sale transactions per hour, with 2.5 petabytes of financial transaction data. I doubt their point-of-sale executable is more than a few megabytes, just like game executables. The Large Hadron Collider generates 40 terabytes of sensor data per second. They do a lot of processing to handle it, but again, the executables probably aren't large. A commercial jet generates 10 terabytes of data every 30 minutes of flight time, most of it is displayed to the pilot continuously, then discarded.

It is all about the data. There is lots of it, everywhere.

Edited by frob
Typos. Typos everwhere. Time for bed.

Share on other sites

It is not always the case that the entire program code is loaded into memory

Oh really? Why is this so?

Share on other sites

It is not always the case that the entire program code is loaded into memory

Oh really? Why is this so?

Because the OS has the ability to do memory mapping.

http://en.wikipedia.org/wiki/Memory-mapped_file

This way it can intelligently load the parts that are actually used, while setting aside the memory addresses for the whole thing if needed.

Share on other sites

The executable itself along with all the associated libraries required less than 45MB, or just under 2% of the total size.

That is strange. My game is quite small in content. But the executable alone is 85.4 MB! Should I not be putting my assets(audio, art, game logic in the exe)? I know these files are there because when I extracted the executable java jar file, the folder that contains the mentioned assets appear.

So I must be doing something wrong, if the executable jar file is so big even for a game with small content(2 minutes of doing everything)!

Share on other sites

JARs don't really work like normal executables. They're more zip files that include the executable files along with other stuff. Not sure if it's accepted practice to put your assets in there.

Share on other sites

It is not always the case that the entire program code is loaded into memory

Oh really? Why is this so?

Because the OS has the ability to do memory mapping.

http://en.wikipedia.org/wiki/Memory-mapped_file

This way it can intelligently load the parts that are actually used, while setting aside the memory addresses for the whole thing if needed.

Do all OS from all brands of computers have this? What happens if a programmer coded the game to load the data for map 1 even when the player is at map 0? Asumming the section of the map are traveled linearly in a numerical order. Would the memory mapping still kick in or follow the code the programmer has written for the game?

Share on other sites

JARs don't really work like normal executables. They're more zip files that include the executable files along with other stuff. Not sure if it's accepted practice to put your assets in there.

Oh Interesting. The strange thing is that I seen people make java games and their games have more assets than mine. But their jar file is super small not even hitting the 10 MB mark and their game boots up by clicking the exe while I actually have to extract a folder from the jar rename it to a folder called "src" and then have the game read that folder which makes the file size containing the exe and the folder equal to roughly twice the size of the jar file.

Share on other sites

It is not always the case that the entire program code is loaded into memory

Oh really? Why is this so?

Because the OS has the ability to do memory mapping.
http://en.wikipedia.org/wiki/Memory-mapped_file
This way it can intelligently load the parts that are actually used, while setting aside the memory addresses for the whole thing if needed.

Do all OS from all brands of computers have this? What happens if a programmer coded the game to load the data for map 1 even when the player is at map 0? Asumming the section of the map are traveled linearly in a numerical order. Would the memory mapping still kick in or follow the code the programmer has written for the game?

Memory mapped files are a feature that has to be explicitly coded to use. So it works however the programmer write it to work.

For automatic swapping between harddisk and memory, a swap/page file is used, which depending on the OS can be a special file or a hard drive partition. Yes, all Operating Systems have paging, modern and not so modern. It isn't aware of what "map" the player is on, but it tries to swap out the least recently accessed memory first, with some intelligence to achieve application fairness. With modern physical memory sizes, paging isn't called for as much, although the same infrastructure is used to initially load application memory. Edited by King Mir

Share on other sites
For Java, if you are looking for a better comparison, there are tools that compile into a native executable rather than into .class files.

Normally the .class files are basically re-compiled on the fly into the real executable through JIT. There are a few free tools to do it, like GCJ (part of the GNU compiler collection, just like their c or c++ compiler, but instead for java), but most tools appear to be non-free. There are pros and cons to doing this, one big benefit is that you can get more aggressive optimization for faster runtime results, but at the cost of tying it to a specific architecture and you lose the run-anywhere aspect for PC, Mac, Linux, and other systems.

When you talk of executable size, it is different from the size of a .jar file that has all the assets embedded in it. As you probably know, a Java Archive (.jar) file is just a .zip format archive file that requires some specific files to be first. You can use almost any compression tool to make the archive and add any data you want, but for convenience just rename it to .jar when you are done. Executable jars are a tiny executable wrapper that have the archive file attached to it.

In Java 6 and before you could use any valid 32-bit zip file, meaning up to 16-bits (65535) individual files with a maximum size of 32-bits (4GB). So if your modern game would ship on a 4GB DVD you could just stuff everything into a giant .jar file and run it. Starting in Java 7 the 64-bit zip format is used, so you can store up to 16 exabytes in your archive assuming you can distribute it.

So yes, "jar size" and "executable size" are very different, even when you have an executable jar.

Share on other sites

I actually have to extract a folder from the jar rename it to a folder called "src" and then have the game read that folder which makes the file size containing the exe and the folder equal to roughly twice the size of the jar file.

You can use the JarFile class to open your own file, search through the file listings for the right JarEntry, and then open it as an input stream decompressing it as you stream the data out the same way any .zip decompression library works. For some resources this can happen automatically. No need to decode the file out to disk unless you have some disk-based need.

Share on other sites

I actually have to extract a folder from the jar rename it to a folder called "src" and then have the game read that folder which makes the file size containing the exe and the folder equal to roughly twice the size of the jar file.

You can use the JarFile class to open your own file, search through the file listings for the right JarEntry, and then open it as an input stream decompressing it as you stream the data out the same way any .zip decompression library works. For some resources this can happen automatically. No need to decode the file out to disk unless you have some disk-based need.

It turns out the JarEntry is another Java class built in to the Java Standard Library. So with the above mentioned Java classes, the jar will just read the files embedded in the jar exe?

One thing I should mentioned is that I am using FileInputStream. Here is my Java game code for reading the art assets.

image = ImageIO.read(new FileInputStream(new File(animations[i])));


Does the above code being FileInputStream have any effect on when my java jar exe not running when I double click on it (unless I extracted the folder from the jar exe)?

Share on other sites

It is not always the case that the entire program code is loaded into memory

Oh really? Why is this so?

Because the OS has the ability to do memory mapping.
http://en.wikipedia.org/wiki/Memory-mapped_file
This way it can intelligently load the parts that are actually used, while setting aside the memory addresses for the whole thing if needed.

It can also happen completely independent of the OS.

For example, Final Fantasy 7 (Playstation version) has code broken up into several modules (menu module, battle module, etc.), and at any given time only the relevant modules are loaded in RAM (i.e. the menu module is loaded when the player opens the menu, and immediately unloaded when the player closes it). The only module that is always loaded is the one that is responsible for loading other modules and booting the game.

Share on other sites

Memory mapped files are a feature that has to be explicitly coded to use. So it works however the programmer write it to work.

For automatic swapping between harddisk and memory, a swap/page file is used, which depending on the OS can be a special file or a hard drive partition. Yes, all Operating Systems have paging, modern and not so modern. It isn't aware of what "map" the player is on, but it tries to swap out the least recently accessed memory first, with some intelligence to achieve application fairness. With modern physical memory sizes, paging isn't called for as much, although the same infrastructure is used to initially load application memory.

While memory mapped files are handled entirely in code, swapping between hard disk and memory is done with support from the CPU itself.  If you try to access part of memory that is paged to disk, the CPU will 'throw' an exception, telling the operating system that the piece of code needs to be moved back into RAM before execution can continue.  Exceptions in a CPU are thrown whenever certain 'bad' situations arise, like for instance division by zero.  In user space, these exceptions are recoverable, meaning that the operating system terminates the program and tells you that the program did something wrong.  If this happens in a core system driver, or in the operating system itself, you will get what is the equivalent of the blue screen of death on Windows, depending on the operating system.

Edit: I wrote that exception are thrown in 'bad' situation in quotes, because not all exceptions are bad.  Hardware interrupts, for instance behave very much like exceptions, but are just signals to tell the operating system to handle 'urgent' situations.  In either case, the processor immediately stops what it was doing to handle the situation before it returns to continue where it left off before the interrupt/exception.

Edited by aregee

Share on other sites

You can use the JarFile class to open your own file, search through the file listings for the right JarEntry, and then open it as an input stream decompressing it as you stream the data out the same way any .zip decompression library works. For some resources this can happen automatically. No need to decode the file out to disk unless you have some disk-based need.

It turns out the JarEntry is another Java class built in to the Java Standard Library. So with the above mentioned Java classes, the jar will just read the files embedded in the jar exe?

One thing I should mentioned is that I am using FileInputStream. Here is my Java game code for reading the art assets.

image = ImageIO.read(new FileInputStream(new File(animations[i])));
Does the above code being FileInputStream have any effect on when my java jar exe not running when I double click on it (unless I extracted the folder from the jar exe)?
Again, a jar file is just a compressed archive file.

When you are running it (by clicking on it or debugging it or any other method) the operating system usually locks the file so it cannot be modified, but you absolutely can read from the file all you want. When the OS locks a file like that other processes can still read from it assuming they have permissions.

Java provides multiple ways to read data from an archive. All of them use a stream-based readers because the archive format uses stream-based compression.

Probably the most commonly used is Class.getResourceAsStream(filename); which finds your file inside the archive and returns your stream directly from the archive. You will need to properly name your file to account for the way Java handles paths inside a jar, perhaps "/com/myname/image.jpg", but it should work just fine. You could provide that stream to your ImageIO.read() function and it would stream just like any other input stream. No need to make a second copy of the data to a temporary file.

For a little more complex functionality you can create a JarFile object passing in your jar file name, loop over all the JarEntry items (those represent the individual file streams) and when you find the one you want, run the line: InputStream input = myJarFile.GetInputStream(myJarEntry); Like the example above, it returns your stream directly from the archive. It is more flexible because you aren't limited to your own file, you can put a different file in as the target for JarFile assuming you have appropriate permissions. Also because you can enumerate over all the different file streams you aren't required to know the exact file name in advance. But if you do know the file name and it is part of your own archive, the result is functionally identical to the Class.getResourceAsStream() version, giving you a stream within your own archive.

You can also open the file as a zip archive, find the entry using exactly the same process as the JarFile/JarEntry search, and end up with a stream.

Or open the file as a regular binary file and process it yourself, the file format is well-documented. This method also works at finding files that don't show up on the archive's manifest, since it is possible to include unnamed file streams and not include them in the listing, or to mark a file as deleted by appending a new listing that doesn't include the removed file. In both cases the data is still present, it just doesnt show up directly by name.

There are lots of options.

Share on other sites

Probably the most commonly used is Class.getResourceAsStream(filename); which finds your file inside the archive and returns your stream directly from the archive. You will need to properly name your file to account for the way Java handles paths inside a jar, perhaps "/com/myname/image.jpg", but it should work just fine. You could provide that stream to your ImageIO.read() function and it would stream just like any other input stream. No need to make a second copy of the data to a temporary file.

I will see if I can get this part to work. It definitely seems to be the easier way to do it concept-wise. Thanks Frob!