So whats a good way to detect a corrupt savegame file?
I ran into a corrupt savegame file this morning while playtesting Caveman.
http://www.gamedev.net/blog/1730/entry-2258672-caveman-v30-general-desciption/
thanks to round robin save and round robin autosave, i had three other good recent saves to chose from.
but the only indication that the file was corrupt was overflow of hut and cave indexes after loading the CRH (cave / rockshelter / hut) database.
perhaps a checksum added to the end?
Checksums are certainly a way to check for corrupt save games. Checking for impossible conditions or illegal values are of course another. Checking that sections of your save game is the size they should be, or even if the size given would go beyond the size of the save game file itself. If you have sections with identifiers, you could also check if a sections reported size would pass the beginning of the next section. If you have sizes for sections, you could also check if a section is following immediately after where it is supposed to follow. In all cases, you will want to decide wether you want to try to recover the save game to a sane or reasonable state, if possible, or if you want to reject the save game altogether to be safe. I tend to be paranoid about checking conditions to be sure a save game won't put the game in a weird state.
If you have ever parsed an Amiga Sound Tracker module song, you know what I mean. That is the most horrible "format" ever made, or lack of format, due to differences between versions, and organic expansion during the years it was common to use that format. There were maybe 50 different "Trackers" made, and some were using the same codes for different things, WITH NO VERSIONING or identifier to tell which "Tracker" you made your song, even with the format being the same between versions that does different things. A song will play in almost any tracker, but if you want it to play correctly, you have to do some qualified 'guessing' by seeing different hints stored in the save file. One of these hints is: Is song name or instrument names written in ALL CAPITAL LETTERS? Then you can assume it was a really old version. Of course, it might be just the author actually writing the names in capital letters on a newer version...
Checksums will provide some protection to external corruption, but if your game is the source of the corruption, and that corruption gets computed into the checksum, then the checksum accomplishes nothing.
Checksums will provide some protection to external corruption, but if your game is the source of the corruption, and that corruption gets computed into the checksum, then the checksum accomplishes nothing.
Just out of genuine curiosity, how can a checksum provide protection to external corruption? If I sound critical, I am not, just genuinely curious as I always want to learn more. + for mentioning the case where your game may be the cause of corruption, baking it into the checksum.
Yeah go with a checksum. If you want something a little more robust than a simple "add up all the bytes" method, consider a CRC checksum. It shouldn't be too hard to find a free implementation, but there's one I found here along with an explanation: http://www.barrgroup.com/Embedded-Systems/How-To/CRC-Calculation-C-Code
Just curious, what implications would there be using a MD5 hash instead of CRC checksum as a checksum?
Sometimes when there was no good option for replacement (because nothing like it exists in base game) we would go with party balloons for a replacement object.
LOL! Oh, i love that one! very SIMs! <g>
Just out of genuine curiosity, how can a checksum provide protection to external corruption?
it provides detection, not protection.
write out the entire savegame file, adding up the bytes as you go, or write it out then read in and add up the bytes. this gives you a checksum. the sum of all the bytes in the file added together. you then store the checksum someplace like the end of the file. when you read the file in, you add up the bytes again, and check the result vs the stored checksum. if they don't match, you know the file has been corrupted.
crc checks are similar, but reduce the chance that two erroneous bytes might cancel each other out and leave a simple checksum thinking everything was ok.
there are also methods of error correction coding where all stored values differ by at least 3 bits. so an off by one bit error can be error corrected to the closest legit value. as i recall, EBDIC is an example of this. such codes are commonly used in telecom and networking protocalls, where transmission errors are a concern.
hmm,
power outages are the cause, not program errors - i double checked to makes sure the reads and writes match up. and there was a power outage soon after the file was saved, so it was already suspect.
real data validation would be a lot of work. however it was a basic form of data validation (data structure overflow error message) that first clued me in to something not being right with the file. otherwise i might have continued happily playtesting away with a corrupt game loaded. THAT would have sucked!
crc would be more reliable than checksum.
anything beyond crc is probably overkill.
the game uses round robin saving and autosaving, with 2 slots each. so if one goes bad, i have three good ones to work with.
in the past i've relied on the built-in game editor and playtest controls to fix corrupt / obsolete save games.
trying to automatically recover a saved game would be a lot of work.
probably best to go with crc for detection, and load previous save as the recovery method, discarding the corrupt file. that's what i did when it happened yesterday. but i had to do it manually (rename files to force it to load the older savegame). with crc check, it could do it automatically if a corrupt file was detected.
Just curious, what implications would there be using a MD5 hash instead of CRC checksum as a checksum?
There are none, really. MD5 basically guarantees you will detect accidental corruption with essentially 100% probability, while a CRC checksum will detect accidental corruption with about as good probability (not nearly as good, since it's not cryptographic, but well above 99.9% since it's after all a checksum). The CRC checksum will completely fail at detecting intentional corruption, MD5 will handle that with no issues (MD5's collision resistance is broken, but in this scenario only preimage resistance matters and MD5 is still strong with respect to that, though you could do better). Computing a CRC checksum is slightly faster than computing an MD5 hash, but not by much (this is because while MD5 does a lot of work, most CRC's operate on a per-byte basis and do a complex finite field polynomial multiplication which is rather expensive) An MD5 fingerprint is four times as long as a CRC checksum. Take your pick.
