Can I ask how you handle writing out objects that are runtime-dependent? I was under the impression that things like buffers and Texture objects that require a device context to create are volatile "memory-only" elements. That's one thing I still do in my onLoad methods from the binary format: create the device-contextual buffers via the game's DX11 device before handing off the object reference.

Yes sorry, most of these structures don't require a parsing step. The above structure does perform the pseudocode of [font=courier new,courier,monospace]for each program: program.handle=device.Create(code[program.offset]).[/font] [font=courier new,courier,monospace]free(code)[/font]. i.e. yes, D3D objects must be created and destroyed. However, this is from the D3D9 version of my shader format, so CBuffers specifically don't have to be created via D3D ;)
Side note - on consoles with nicer graphics APIs, this is isn't required (all that's required is pointer-patching to where you streamed the VRAM data instead).

Having written a loader for the 3ds format I heartily recommend avoiding it if possible. It's an old format with limitations that might cause you problems over a newer format.

For example: 1) All filenames are in 8.3 format, and saving a .max file into .3ds normally truncates the filenames if they are longer than 8.3, which can lead to loss of data if you have a prefix. (Learnt that one the hard way).
2) The maximum number of vertices per object is 65536, which might be a problem depending on how high resolution your meshes are.

My current project uses .obj files because they are easy to parse, pretty much every modeller can save them, and I'm still early in development. Pretty soon I'll have to start using a format with more features such as collada, since .obj doesn't support bones or keyframes etc. But my plan is to convert the collada files into my own format during a resource build phase as others have suggested.

Uh, the file format will have zero impact in actual rendering performance, as the model will be converted to the program's internal format regardless of the original format. As for loading performance, it really depends, binary formats are usually the fastest, and also take the least space on disk, but are comparatively much harder to parse.

For instance, many people use .obj not because it is fast, but because it is very easy to parse, readily editable by any text editor, and is usually good enough for most models even if it is quite wasteful in terms of storage. But loading the same model from a .obj and from a .3ds will yield the exact same model representation in your game's memory and there will be no performance difference after loading is complete.

This is not always the case. For example, if there needs to be a file loaded during run time. Such as a cache miss, if you are running low on resources, your cache may not load some resources until it needs it at run time, or may unload something to make room for a different resource. This causes a load during run time and can effect performance. It can be avoided most of the time in a well written cache system but not always.

Even triple A games suffer cache misses occasionally.

So in this scenario a 3d binary format will effect performance far less, than a slower to load exchange format.

So like I said it depends on the internals of what your program is doing under the hood with resources.

[quote name='Bacterius' timestamp='1349243010' post='4986291']
Uh, the file format will have zero impact in actual rendering performance, as the model will be converted to the program's internal format regardless of the original format. As for loading performance, it really depends, binary formats are usually the fastest, and also take the least space on disk, but are comparatively much harder to parse.

For instance, many people use .obj not because it is fast, but because it is very easy to parse, readily editable by any text editor, and is usually good enough for most models even if it is quite wasteful in terms of storage. But loading the same model from a .obj and from a .3ds will yield the exact same model representation in your game's memory and there will be no performance difference after loading is complete.

This is not always the case. For example, if there needs to be a file loaded during run time. Such as a cache miss, if you are running low on resources, your cache may not load some resources until it needs it at run time, or may unload something to make room for a different resource. This causes a load during run time and can effect performance. It can be avoided most of the time in a well written cache system but not always.

Even triple A games suffer cache misses occasionally.

So in this scenario a 3d binary format will effect performance far less, than a slower to load exchange format.

So like I said it depends on the internals of what your program is doing under the hood with resources.
[/quote]

I don't see what cache misses have to do with the storage formats, you don't reload data from disk on a cache miss, the CPU cache pulls data from RAM on cache misses(This is automatic) and the only thing that matters for this is the internal (in memory) format.

If you are streaming data in and out of the system at runtime then yes, you need a efficient storage format and it might even be worth storing the data in a compressed form and decompress it as it loads (depending on how much free CPU resources you have), runtime streaming of data from disk however is only really needed for large open world games (or consoles such as the xbox360 since it has almost no RAM)
If you have a software cache to stream data in and out at runtime you're

I don't see what cache misses have to do with the storage formats

After you've streamed your stored bytes into memory, you've got to do some work with them. Depending on what that work is, and how the file is laid out, you'll get a different amount of cache misses during that work.
[edit]Ah, i see he's talking about a software resource cache, not the CPU's RAM cache... In that case, smaller more compact file formats would allow you to fit more files in your 'resource cache' at a time, whereas large bloated formats would waste memory and fill up your resource budget (requiring less resources to be loaded at once, requiring more streaming).

[quote name='SimonForsman' timestamp='1349333532' post='4986671']I don't see what cache misses have to do with the storage formats

After you've streamed your stored bytes into memory, you've got to do some work with them. Depending on what that work is, and how the file is laid out, you'll get a different amount of cache misses during that work.
[/quote]

The post i replied to though seemed to imply that you'd reload the data from disk on a cache miss. (Maybe i just misunderstood it)

[quote name='Hodgman' timestamp='1349334765' post='4986676']
[quote name='SimonForsman' timestamp='1349333532' post='4986671']I don't see what cache misses have to do with the storage formats

After you've streamed your stored bytes into memory, you've got to do some work with them. Depending on what that work is, and how the file is laid out, you'll get a different amount of cache misses during that work.
[/quote]

The post i replied to though seemed to imply that you'd reload the data from disk on a cache miss. (Maybe i just misunderstood it)
[/quote]

This is the way it works. You can't put everything in memory at a single time for large games, especially open world games, their resource managers ( caches I've been calling them ) work over time to keep what you need in memory and remove what you don't. But they are never perfect in that they often "Miss" and have to wait to load from disk.

Having another thread for loading of uncached resources can help, but then you deal with pop in's and things like that.