performance with random access files

Started by
2 comments, last by GameDev.net 18 years, 3 months ago
Im curious. Is there a performance hit if I switch between reading and writing more often. For instance read block, write new block and so on. Is the switch expensive?
Advertisement
Writing big chunks is definitely way faster than writing lots of small chunks. I got a pretty nifty utility a while ago that showed the transfer rate for various chunks sizes (can't remember what it was called)... After a certain size, it reached the limit of my ATA/100 hard drive which was somewhere around 60 MB/s. (That's pretty fast- you could write a 100MB file in 2 seconds or so!) For really small chunks like 512 bytes or something, the transfer rate plummeted- can't remember what the exact value was but it was on the order of 20-100 K/s.

Hmm, although if you switch between reading and writing of the same 2 files, I'm not sure how that affects performance. Most hard drives now have at least an 8MB cache, so I imagine that would help.
^^^^ What he said.

Actually, for game programming on the PC I find that I'm less limited by the file reads and writes than I am on any other device (most devices I work with have to read from ugly memory interfaces that are slower than optical, and that's saying something).

But yes, you *might* see a performance hit. It's really dependent on where you are doing the reads and writes. You'll find that a lot of things in programming are situational, and that's why benchmarks almost always vary from computer to computer, from application to application.

Typically, block writing bigger chunks is faster because it utilizes a lot of the infrastructure the drive was meant to use, DMA/MMIO/whatchamacolit in your desktop. Smaller chunks typically have problems due to file system overhead (depending on the system of course; Reiser loves small files, NTFS makes me want to bleed).

Overall, unless your application is *seriously* databound, as in, hundreds of multi-meg reads/writes per minute, you likely won't see a difference. Most modern operating systems are extremely effecient about file caching and buffering writes that they seem as smooth and transparent as silk (until you start talking about megs and megs of data, then it starts to congest).

Suggestion: write it both ways and do a small benchmark for yourself. If it's simply easier to write it one way or the other, I'd suggest going that way and not giving it any thought until way later in the process.
Any seek opertation is usually quite expensive. You might consider using memory mapped files if you write/read to random locations within a certain range.

This topic is closed to new replies.

Advertisement