Memory File Mapping?

Started by
4 comments, last by BitMaster 9 years, 4 months ago

Hello,

I'm on the search for file mapping library. I know Win32 has a file mapping functionality, but this doesn't help me for the Unux/Linux community and I know I maybe able to write one my own. (Why reinvent the wheel?) But is their a MIT license file mapping library out their thats cross platform? My project granted will be fine with 4GB RAM machines for small scale projects (32bit), maybe 90GB RAM servers(64bit) if we are talking big data, but supposed someone wanted to go the inexpensive route and do clustering with my application. It will need to use the HD (slow) as means to store data for the program (which maybe a lot of read and writes) if we are talking 5GB to 10GB of data required to be stored.

If someone can give tips or point me in the right direction, that would be amazing, thank you!

-Ajm

Check out my open source code projects/libraries! My Homepage You may learn something.
Advertisement

I'll direct you PhysFS.

It is cross platform, it restricts access to within your file system, it lets you use compressed resources and also mix-and-match from compressed final resources and development resources.

I'm sorry frob,

it's a nice library, but it's not what I'm looking for exactly. (I should have been more specific) I'm looking to basically store records like a RDBM or like MySQL. Where I'll need to constantly require to be able find data, and or write it from small chunks to large chunks in a file/vector array.

Check out my open source code projects/libraries! My Homepage You may learn something.


I know Win32 has a file mapping functionality, but this doesn't help me for the Unux/Linux community and I know I maybe able to write one my own

Linux has mmap() which is the equivalent of windows CreateFileMapping(). This has been available since 2001? and predates the windows equiv.

mmap is also easier to use.

http://man7.org/linux/man-pages/man2/mmap.2.html

You might have a look at mman. It's a small library to provide an mmap-compatible API on Windows.

Tristam MacDonald. Ex-BigTech Software Engineer. Future farmer. [https://trist.am]

Well, there is (obviously) boost::mapped_file. Though I personally dislike boost, so I would (in fact, I have) reinvent the wheel for this simple task. It's not hard to implement the basic functionality, nor a lot of code -- and the advanced POSIX functions (testing in-core, hinting, reordering pages) won't work with Windows anyway, no matter what you try.

A quick Google brings up this one, which is read-only (so not exactly what you need, but 95% good). License looks liberal enough (BSD) and you should be able to make it exactly as you need by changing the file open and memory protection flags in 2-3 locations (should take like 5 mins).

This should be the current documentation of boost::mapped_file. Although Google is sometimes a bit uncooperative there I don't think a documentation that is 19 releases behind is ideal.

This topic is closed to new replies.

Advertisement