Recently I've been doing a lot of work creating file formats for my engine, but everywhere around me I see people moving over to text-based formats, paticularily XML and XML-esque structures. I've never understood this myself. While text-based formats kick ass for debugging (And in some cases extensibility), I just find text formats to be too slow (And large) to deal with in a commercial environment. The main problems I see with text based formats are as follows: - Loading times: In a binary-based format, you can read a primitive value directly into memory, with at most a file read, an assignment and an endian-swap if required. With text-based formats, you'll need to convert the text to the required type, which is a comparitivly expensive operation. The number of digits you require can take its toll on that as well, as in binary you'll normally be reading 4 or 8 bytes regardless, whilst with a text-based format you'll be reading a variable amount depending on the size of value. You also have to deal with skipping over whitespaces, parsing block headers and the like. This can take a substantial hit on loading times, and now that streaming is becoming the norm (Which requires lightning-fast reads to be effective), it seems like a step back. - File size: Pretty much the same as above, storing numbers as text generally takes up more space than if you just wrote out the value directly. You also have to deal with plain-text headers, whitespaces and syntax characters. For a comparison, I've got a dual binary/text file format, and exporting the same model in binary was about half the size of the (Incredibly vanilla) text-based alternative. While hard-drives and storage mediums are getting a whole lot bigger, a saving of ~25-50% is certainly nothing to be scoffed at. - Accuracy: Floating-point numbers can be a real pain, but at least with a binary-based format you'll always get out the exact same value that was put in. With text-based formats, you'll probably have to limit the number of values in the mantissa so you don't have to read in 30 characters for a tiny value. Whilst the difference between 0.000000005 and 0.00000001 is pretty damned minimal, it can have an affect on objects with a paticularily small scale or where extreme accuracy is required (Very, very rare, but I have seen it happen). Although that's not to say I don't recognise the benefits of text-based formats: - Easy to debug: Debugging file-formats can always be a total pain the ass, especially when a lot of floating-point numbers are involved (Converting hex to IEEE in your head is something I doubt many people have mastered). Text formats make it a lot easier to verify, and change during testing. - Extensibility: While it's not an implicit property of text-based file formats, the nature of them promotes a more ordered structure where objects can be identified by unique strings, which allows programmers to add new elements and information without breaking the format. - Community-friendly: For games that thrive on active mod communities (Such as Doom 3), a text-based format allows even fairly inexperienced programmers to write importers/exporters for their favorite programs with reasonably little effort. Binary formats, on the other hand, can be pretty much illedgible without a rosetta stone. So text-based formats certainly have their uses, but I don't believe they're the way forward for all formats. Config files should always be text (Unless of course the system is exceptionally tight on memory/storage) so that non-programmers can tweak the game settings easily, and users can customise the game to their liking. 3D file formats, level files and animations, on the other hand, have little to be gained from moving to a text-only approach. They give larger file sizes and increased loading times, and the main benefit of a text-based approach (Easy editing) is fairly moot as it's incredibly hard for the human mind to understand a complex object given only numerical coordinates, let alone edit it with any sort of real comprehension for what they're doing. So what are your thoughts on this? Do you believe in the push for text-based formats is a good thing? If so, why?

in most cases i think, it´s desirable to sacrifice some performance in order to debugg better

although its only reading and writing performance which is lost ... if you want you can make cachefiles out of your xml data, in order to load them faster the next time?

as for the numbers: saving in hex make it pretty fast to load into a programm (at least faster than with base 10)

and if HD-Speed is your bottleneck, try zlib

Quote:Original post by Joakim_ar
ronnybrendel: Saving floating point values in hexadecimal kinda defeats the purpose of XML being easy to read. And besides I don't think the processing time required to parse a number means anything compared to the disk access.

your right. for integers it is a bit faster, and it requires less space in most cases

but you´re flexible with the size of your number 1-8 bytes for 32bit e.g. ... i mean it could eat less memory than binary (so you can say that loading a fully charged binary integer maybe as slow/fast as loading a hexvalue --- depending on the number itself) - and you can read it with your own eyes ;)