Hello all, I am in the process of designing a new model format for my own personal engine (although the fact that it is for models doesn't really matter). In the past, I've worked for a few game companies and everyone seems to have their own methods. One of the best I've come across was loading compressed files into a temporary memory buffer, allocating space for the decompressed data, then decompressing and directly using the data. The file format used a relocation table in order to fixup internal data to be pointers into itself, so it could be used as-is. When the model was no longer needed, only a single free was necessary to remove the entire model heirarchy tree, vertex data, etc. Plus, the data was compressed, and read all at once, so slow media reads were minimized. This is all well and fine if you have 32-bit pointers (they were using unsigned longs for relocation space), but if I decide to support 64-bit Windows natively, my pointers aren't going to fit anymore. Besides, VC gives lots of warnings for casting stuff between void * and u32... I could increase the space in the file to 64-bit, but it seems overkill -- especially if the file format is cross-platform, and I don't see any console using 64-bit pointers in the near future. :/ So basically what I'd like to know is, how do you guys do it? Do your formats still live in a 32-bit world, or is there some other neat trick that I'm not aware of yet? :)

It's probably a bad idea to store native pointers in a file format designed for cross-platform use. A portable file format should specify all the data in it in a way that doesn't depend on byte orders, word sizes, pointer sizes, etc. Of course this means that translating from the file format to a more convenient data structure and vice-versa could involve some more complicated interpretation of the file. If you're going for portability, neat tricks should probably be avoided.

IMHO the solution with the lowest effort is to use arrays of objects, and to refer to objects by explicit indices and implicit a special object with all the base pointers to the belonging arrays. So the relocation stuff is restricted to compute the new base pointers, and the entire file could be loaded into memory as one big block.