File formats

Published August 23, 2004
Advertisement
Meh. Should have switched to a VFS like PhysicsFS a long time ago. It has vastly simplified quite a few things.

For one thing, all along I have had a custom file format for my Golem tilesets that contains all of the data for a tileset (actual tile graphics, random distribution tables for selecting a random tile from a set, data for setting the collision characteristics of a tile, etc...) all compacted into a binary file to be opened by the tileset loader when needed. Thing is, it's a colossal pain in the ass to do it this way, really. I hacked a custom version of a Lua interpreter with a set of special functions for handling data in my various formats, and to build a tileset I write a script which will collect all the tiles and data and pack it all together to construct the binary file. Making changes, even miniscule ones such as modifying the random characteristics of a set, requires making the change in the script then rebuilding the file. Not too onerous for the most part, but a big PITA when I am tweaking.

Since the final assets will be archived into a pack file, however, it seems redundant to me to pack a tileset into a custom file, then pack that file into the data archive. So I re-implemented tileset loading to load from a nested directory structure as an alternative to loading from a custom file format. Since the VFS can view a directory tree or a ZIP archive transparently as a filesystem, in the final distribution version I can just ZIP up the data directory into the data archive file, and the tileset loader can still load directly from the nested directory structure inside the archive. I can tweak things a lot more easily this way, without the extra step of rebuilding the tileset file.And to protect my precious, precious data ( =D ) I can simply write my own extensions to PhysicsFS to implement a custom archive file format or possibly to support encryption. Or if I decide to make it easily moddable by the player, I can just leave it as a plain ZIP archive and let them unzip, modify, and re-pack at will, without having to do a bunch of custom scripting to make their changes.

It's little stuff like this that I continue to learn as I go. Maybe someday, I'll actually be a real game programmer. ;)

Slowly, ever so slowly, the weather seems to be cooling off. It's supposed to get down into the 90s tomorrow, thank the light. Still a little warm for my liking, but more tolerable than the high 110s we've been getting. I may, just may, survive living in the desert after all.
Previous Entry PhysicsFS
Next Entry New tutorial online
0 likes 3 comments

Comments

evolutional
I'm looking for a VFS system to use, I was considering using my own but I feel sure there's a better option out there... Do you recommend PhysicsFS? What's the API liken for using/integrating with an existing project?
August 24, 2004 02:44 AM
JTippetts
I'm still evaluating PhysicsFS, so I won't exactly recommend it, but so far it has worked without any major troubles.

It is implemented in a very C-like manner, so it is simple to update file functions that use FILE pointers to use PHYSFS_file pointers instead. Converting existing code that uses streams is a little trickier.

There are a few things that irritate me offhand about the library. One is that the PHYSFS_read and PHYSFS_write functions, which mimic FILE fread and fwrite, changed the order of parameters. Whereas fread/fwrite accept parameters as (buffer, size, count, FILE *), the PHYSFS versions accept them in the order (FILE, buffer, size, count), requiring a little more work to convert existing file code over to the VFS. It's minor, but it's irritating, and if I hadn't already encapsulated most of my file stuff, I might have chosen a different library. As it is, I didn't have a whole lot of code to switch over.

The relative lack of text file support is a little irritating as well. PhysicsFS opens all files in binary format, and does not implement any fscanf() type functionality, so I've had to implement my own code for reading string tokens and such. A lot of my data relies on being able to read settings or filename lists from index files, so I sort of need this sort of functionality. If I weren't so damned busy, I might consider implementing an fscanf() and submitting it to the maintainers, but I really have too many things on my plate as it is.

Other than these little issues, the library works great.
August 24, 2004 07:22 PM
evolutional
Thanks for the response. I downloaded PhysicsFS last night and added it to the basic shell of my project for testing purposes. It seems to do what I want for now, however I am a little put off with the naming scheme (PHYSFS_*). I'll probably wrap it in some classes to emulate my existing file handling (non-VFS). Another possible problem needing a workaround is that I use TinyXml to load and save my XML documents, obviously this isn't compatible with the PhysicsFS so maybe a hack of one of the libraries is in order to make this work. It may not even be a problem, being that the only times I'll really want to save a file is during editing or player saving, so these files wouldn't need to be added to the Zip.

Anyway, thanks for the head's up on this library, I'm sure it's going to prove useful.
August 25, 2004 02:15 AM
You must log in to join the conversation.
Don't have a GameDev.net account? Sign up!
Profile
Author
Advertisement
Advertisement