Memory mapped files
What are you trying to do? Pack your data into a big block of resources? You might want something like resource.index and resource.pack, index containing the indexes into the resource.pack file? If you are smart you can make a program that packs your data, and creates a header file instead of an index file, and you can compile with the header file (containing offsets, data formats, and such). Why would you need memory mapping unless you are sharing across processes?
That's about right. And that header thing was what I was thinking of doing and maybe if I wanted a bit more pain, throw in some compression, but that could be bad for speed. I was thinking memory mapped because I'd need to play music and play sound effects at the same time if both such files were in the one resource file. Or maybe it'd be better to have a resource file for each type of resource then I'd need several threads rather than memory mapping? Or something? Which is why I was hoping someone could show me where the information is for my research...
There are a lot of reasons to use memory-mapped files. Speed is one of them. Reading a file via memory-mapping can easily outperform traditional buffered i/o methods. The reasons:
1) Memory-mapped files do no copying internally. Typically with buffered i/o data is read from disk into a buffer, then copied into another buffer when the application calls read(). Memory mapped files skip this step by reading the data directly into the area of memory that you access it from.
2) Memory mapped files work in page sized chunks of data. On Intel machines this means the fundamental unit is 4KB. Plus you can give the memory-mapped file API hints as to how you use the file (sequentially or random access) to optimize how it reads the data.
3) Memory mapped files use the same code in the kernel as page-swapping for virtual memory. This code is tightly optimized because the whole OS depends on it (while I'm sure buffered I/O is also tightly optimized, the extra copies and memory allocation still slow it down).
1) Memory-mapped files do no copying internally. Typically with buffered i/o data is read from disk into a buffer, then copied into another buffer when the application calls read(). Memory mapped files skip this step by reading the data directly into the area of memory that you access it from.
2) Memory mapped files work in page sized chunks of data. On Intel machines this means the fundamental unit is 4KB. Plus you can give the memory-mapped file API hints as to how you use the file (sequentially or random access) to optimize how it reads the data.
3) Memory mapped files use the same code in the kernel as page-swapping for virtual memory. This code is tightly optimized because the whole OS depends on it (while I'm sure buffered I/O is also tightly optimized, the extra copies and memory allocation still slow it down).
As for references, aside from the SDK docs I reccommend Jeremy Richter's "Advanced Windows". It has a chapter on memory mapped files, and in general is a treasure trove on how to take advantage of features in the Win32 kernel. Definetly worth picking up.
------------------
-vince
Thanks. I had such thoughts on memory mapped files, but needed a bit more detail. I'm getting ahead of myself, but is this principle more or less the same if I were to port to Linux? I guess yes, as an OS swap file is an OS swap file and a memory mapped file appears to be the same principle...
The principle is the same. As far as the implementation, I'm not familiar with it, so your mileage may vary...
------------------
-vince
Author
Hi,
Could someone point me in the direction to information on memory mapped files or provide information for, please? If I haven't RTFM, I apologise for my blindness. Anyway, what are some alternatives to such things as that's all I've thought up for a data file to hold all the game resources and stuff (sfx, music, gfx, scripts, etc). Unless I'm approaching this all wrong of course...
Could someone point me in the direction to information on memory mapped files or provide information for, please? If I haven't RTFM, I apologise for my blindness. Anyway, what are some alternatives to such things as that's all I've thought up for a data file to hold all the game resources and stuff (sfx, music, gfx, scripts, etc). Unless I'm approaching this all wrong of course...
Vince,
thanks for the explanation - I had thought memory mapping was really just for multi-process work.
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
