Sign in to follow this  

performance with random access files

This topic is 4353 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Writing big chunks is definitely way faster than writing lots of small chunks. I got a pretty nifty utility a while ago that showed the transfer rate for various chunks sizes (can't remember what it was called)... After a certain size, it reached the limit of my ATA/100 hard drive which was somewhere around 60 MB/s. (That's pretty fast- you could write a 100MB file in 2 seconds or so!) For really small chunks like 512 bytes or something, the transfer rate plummeted- can't remember what the exact value was but it was on the order of 20-100 K/s.

Hmm, although if you switch between reading and writing of the same 2 files, I'm not sure how that affects performance. Most hard drives now have at least an 8MB cache, so I imagine that would help.

Share this post


Link to post
Share on other sites
^^^^ What he said.

Actually, for game programming on the PC I find that I'm less limited by the file reads and writes than I am on any other device (most devices I work with have to read from ugly memory interfaces that are slower than optical, and that's saying something).

But yes, you *might* see a performance hit. It's really dependent on where you are doing the reads and writes. You'll find that a lot of things in programming are situational, and that's why benchmarks almost always vary from computer to computer, from application to application.

Typically, block writing bigger chunks is faster because it utilizes a lot of the infrastructure the drive was meant to use, DMA/MMIO/whatchamacolit in your desktop. Smaller chunks typically have problems due to file system overhead (depending on the system of course; Reiser loves small files, NTFS makes me want to bleed).

Overall, unless your application is *seriously* databound, as in, hundreds of multi-meg reads/writes per minute, you likely won't see a difference. Most modern operating systems are extremely effecient about file caching and buffering writes that they seem as smooth and transparent as silk (until you start talking about megs and megs of data, then it starts to congest).

Suggestion: write it both ways and do a small benchmark for yourself. If it's simply easier to write it one way or the other, I'd suggest going that way and not giving it any thought until way later in the process.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Any seek opertation is usually quite expensive. You might consider using memory mapped files if you write/read to random locations within a certain range.

Share this post


Link to post
Share on other sites

This topic is 4353 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this