Jump to content
  • Advertisement
Sign in to follow this  
ddengster

Is fread considered a seek memory on disk operation?

This topic is 670 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

So I've been trying to read from a simple pak file, and I have 2 options on how I want to read one of its internal files on request.

 

1) Fread the entire pak file once at initialization, copy the entire pak file into ram, then load the request data on request.

2) Fopen the pak file and fread the manifest data at initialization, then do fread the requested file data with the offsets acquired.

 

Number 1 doesn't seem to be the reasonable choice when it comes to large files. Number 2 seemed to be the correct choice, but it got me wondering: is fread a read from disk operation? If so isn't it pretty slow? How does (at least) windows do this?

Share this post


Link to post
Share on other sites
Advertisement

Yep, memory mapping is the correct approach. Note that this still involves reading from the disk. It can still be beneficial to issue large memcpy operations with this approach, rather than bouncing around reading a bit at a time.

 

is fread a read from disk operation? If so isn't it pretty slow? How does (at least) windows do this?

What could it be other than a disk read operation? There is a disk cache in Windows, and there are some things you can do to help Windows do a better job. Same goes for the memory map setting, in fact. Games deal with the question in a particularly harsh setting when loading from optical media. Ordering of files matters in that case, to reduce seeking (which is really costly on optical) and improve cache behavior.

Edited by Promit

Share this post


Link to post
Share on other sites

There is some buffering that takes place behind your back, as covered somewhat above.  For reading small files there are buffers by the OS and by language libraries. The exact details vary by system and libraries, anywhere from a few kilobytes up to a megabyte or more.

 

But memory maps have an even cooler feature: since you are declaring broad intentions rather than tiny pieces, the OS can schedule and re-order disk reads for the entire range. This is even more significant on spindle-disks than SSD disks.  So if the OS knows data is scattered around in twenty different places the OS can read them in the order that is optimal for the drive, picking them up as it sweeps across rather than sequentially which may require jumping back and forth a few different times.

 

For large files memory maps are typically the best solution, even though they tend to require OS-specific functionality to implement (or the C++ Boost library that seems to have a little of everything).

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!