I'd like to flush the savegame to disk. That way the file won't be lost or corrupted if the power goes out soon after saving. Just trying to make the code more bulletproof.
How would you interpret the following docs?
Stream I/O, has links to fopen, fclose, and fflush:
If I understand them correctly, I can just call fclose. After all, it releases windows file handles, which I would assume flushes OS write buffers to disk..
But I may need to open the file with "wbc" instead of "wb" =OR= link the "commit to disk" obj file, and call fflush before fclose.
First, there is a good way to it, but that isn't a direct answer. So the direct answer first.
You can attempt it, even flush and close the files, but that doesn't guarantee anything.
A few years back (looking to find the article in another tab) there was a study of various disk drives. It turned out that most of them ignored the SATA and SCSI commands to flush their write cache.
I'm not seeing the article, but basically...
The runtime library buffers your write requests. When you flush your stream you are asking the runtime library to give all the data to the operating system.
The operating system buffers your write requests. Many operating systems provide commands to flush or sync these buffers.
The hardware buffers your write requests. Both SCSI and ATA interfaces have instructions to flush or sync the buffers.
A few years back (2004-2006 era?) there were some comparisons of disk performance, and the study authors found that nearly every HDD on the market either ignored or postponed commands to flush their buffers, nearly all of them returning immediately that the buffers had been flushed, a very small number did the recommended behavior of actually flushing the buffers and then returning back that the flush was complete.
Now, on to what you can do.
Most experienced programmers are used to the "copy and swap" idiom. It has been around for ages, and disk data integrity was one of the main reasons for it.
When applied to objects in memory, the copy and swap idiom means that you fully construct a temporary object and do the operations on the copy. If there is an error during construction, you can discard the incomplete copy and the original version remains intact. After the temporary object is fully built without error you swap the old contents and the new contents.
It works similarly for disk files. Write your data to a temporary file on disk. (For example purposes, "myfile.new") If there is any error while writing the file you can notify the user and delete the temporary. After you have completely written your save data and know it is valid, you rename the old file to something unique (realfile.txt -> realfile.old), rename your new file to the proper name (myfile.new -> realfile.txt) , and delete the old file (realfile.old).
If something goes critically wrong during the process, such as a power loss, the damage is mitigated. If the error happened while the file was being built the old file still exists (myfile.new and realfile.txt) you know the save was incomplete and can potentially recover some contents. If the error happens after the file is written but during the renaming process there will be two files (realfile.old and myfile.new), again you can recover easily because you know one file (myfile.new) is valid. If it failed with the new data renamed but the old file mid-delete (realfile.old and realfile.txt) you can finish deleting the file (realfile.old). A common example of this happens in Microsoft Office, you see all those temporary "filename.~oc" files which the program keeps locked; they are following this pattern using the "~" for the intermediate file.
If you need an even more secure system, provide three copies that must each be replaced. When there is an error one of them will be invalid and the other will still be correct. You can also reduce it to two live copies and one dead copy so the space can be reused for something else. I have seen this used in save games on certain cartridge-based systems before. Using uppercase and lowercase to denote versions, basically you have copies occupying three spaces (ab-), then you write new data to C (abC), when it is complete you write to another slot (AbC), and when that is complete you mark the remaining file as dead (A-C). If something happens to corrupt the data, like the user pulling the cartridge, you always have a copy you can rely on. If you have (abC) you notify the user that the previous save was incomplete and verify+load whatever is in slot a and b. If you have (AbC) or (?bC) you load the later stamped of the two, then replace the ? with the new version, then kill the old one. If you have (AbC) you verify+load whatever is in A and C, then mark b as dead. Then you can use the dead slot for something else, such as a volatile quicksave slot.
If I understand them correctly, I can just call fclose. After all, it releases windows file handles, which I would assume flushes OS write buffers to disk
Nope. The OS and even hardware are more concerned with performance. Flushing writes to disk is _slow_. It not only makes your app slow, but makes all the other apps using the disk slower, too. A few years ago there was a weird behavior in Firefox that could cause a particular OS to hang for an entire 30 seconds due to disk flushes.
If you want better save files, then (a) save them in rotation and (b) save them often and (c) enable cloud saves/backups. Never, ever, _ever_ modify a save file after it's written; always write a new one and then delete the old one only after you've finished writing the new one (or better, rename the old one to a .backup## filename and don't try to delete it right away).
Sean Middleditch – Game Systems Engineer – Join my team!
It turned out that most of them ignored the SATA and SCSI commands to flush their write cache.
Lovely!
Well, i'm already doing a round robin save to new file, then copy over the older of the last two saved game files. Even if it fails, you still have the last savegame. Was just hoping to add flushdisk to it for thoroughness. So much for that idea! : P
According to the SQLite guys, http://sqlite.org/transactional.html their implementation is well tested against such horrors. Maybe if this is true and you want some additional security, you might consider using a SQLite database to save your data.
Just an idea. I have no knowledge whatsoever about harddrives and flushing and so on.
In these situations, caching can be turned off. This is done at the time the file is opened by passing FILE_FLAG_NO_BUFFERING as a value for the dwFlagsAndAttributes parameter of CreateFile. When caching is disabled, all read and write operations directly access the physical disk. However, the file metadata may still be cached. To flush the metadata to disk, use the FlushFileBuffers function.
Some applications, such as virus-checking software, require that their write operations be flushed to disk immediately; Windows provides this ability through write-through caching. A process enables write-through caching for a specific I/O operation by passing the FILE_FLAG_WRITE_THROUGH flag into its call to CreateFile. With write-through caching enabled, data is still written into the cache, but the cache manager writes the data immediately to disk rather than incurring a delay by using the lazy writer. A process can also force a flush of a file it has opened by calling the FlushFileBuffers function.
But I don't know how that affects performance and whether it always works.
EDIT: If it is Windows only, you are probably better using the native file IO functions (CreateFile, WriteFile, etc.) than the C lib ones (fopen, fwrite, etc.) anyway. If you need to go cross-platform I'd look and see if there is a boost library that does what you want.
"Most people think, great God will come from the sky, take away everything, and make everybody feel high" - Bob Marley
Well, I looked over all the docs again, and it looks like I don't really have to worry about it. I don't leave the file open. I open the file, write all the data, then immediately close it. At that point the OS flushes the buffers to disk. So no extra flush instructions are required.
