In order to import 3d models, a 3d model file (i.e. COLLADA) and some maps are needed.
Having all that data out there in some folder isn't too smart because it can be viewed or even worse, edited, easily by anyone, so I noticed that many games have their data packed in one or many large files.
How do game engineers do that? Any thoughts on this matter?
Having all that data out there in some folder isn't too smart because it can be viewed or even worse, edited, easily by anyone
To pile on, you're right that editor-friendly formats aren't a great idea, but its not because how easy they are to edit, its because editor formats are built for editing and don't have any concern about hitting soft-real-time requirements, or how small they need to be to suit disk-bound or digital distribution.
The ideal formats for AAA and very demanding games are minimal processing between reading and seating them in memory -- data should, per-platform, be read into its exact in-memory format, complete with proper padding, alignment, and data-representation, when possible (zero-out the padding, for security and better compressibility). That allows data to be round-tripped with no intervention in theory, in practice, you'll probably want to do a minimal amount of validation.
For games that aren't as demanding, typically digitally-distributed ones (Indie games typically fall into this category, though not all that do are indie), file size and simplicity are often better goals to strive for. For that, you'll probably want a binary file format that's a sequential representation of the in-memory format, but which removes padding and ignores alignment. Then, you'll probably want to compress the entire packfile or whatever with something like zlib. For some file types, like textures, that have specific, effective compression like DXT and friend, you'll want to do that before compressing further, or maybe use a scheme that only compresses files without specific compression methods.
This is a very important point.Games pack files for speed and for reduced install size.
[...]
Pack files are used because they're _faster_, not because they're more "resilient" to those evil horrible hated detestable end-users tinkering with the bits and bytes.
In general, people will tinker with and change the assets for your game if they love it.
So, even if you pack and compress or align your game data, consider making a small sdk with the tools so that people don't have to be ingenious to get at it. :)
Those tools includes pack readers, model viewers, etc.
isn't too smart because it can be viewed or even worse, edited, easily by anyone
This sentence almost made me vomit.
This is a screenshot from when I was tinkering with Final Fantasy VII assets when I was 17. Assets I hacked out of the game.
Today I work for Square Enix as a senior graphics programmer on Final Fantasy XV.
Why is it that you think having your resources hacked is a bad thing? Hacking Final Fantasy VII and getting encouraging results made my career.
L. Spiro
This is a nice article on prepping data offline ready for efficient loading: http://www.gamasutra.com/view/feature/132376/delicious_data_baking.php?page=1
Another thing to think about is whether you want to use a 'push' or a 'pull' approach to loading resources.
The pull approach is perhaps more natural, you have a bunch of resources set out into a logical (to a human) folder structure (possibly in some sort of zip-esque archive format), then your game code requests the assets it needs in some arbitrary order and they get loaded. Sometimes when you're loading a model, it might contain dependencies on some texture or sound, so you fire off loads for them too.
The problem with the pull approach is that you're seeking all over the place. Which is fine for solid state storage (SSD, cartridges, mobile) is OK for hard disk drives, but is a total nightmare for DVD/Blu-Ray, etc, as the seek costs can be huge.
The push model is an approach where you create a single file for a collection of stuff that will get loaded together. So if your game is level based and non-streaming, your tools might construct a single level file containing all the appropriate models, sounds and textures. The game code will request to load the level file, and the load process will push each of the resources it comes across into the right subsystem.
I guess with the current crop of platforms, and the fact that games stream so much dynamically, the push vs pull choice is less important, and it's always possible to make the pull model work OK by carefully laying out resources on the storage media, but I think the push approach is the better one if you're planning out a system from scratch.
If you're using C or c++, look into physicsfs.
It allows loading resources from compressed archive formats such as zip, 7z, and wad. It acts like a filesystem so you can stream files from it or just load them whole into ram. Definitely worth the effort to integrate.
Firstly, you probably don't want an interchange format like COLLADA or the native format of your modelling package in your final game; it has all the data needed for editing your model, and a lot of that simply isn't needed in the final game, so you'll usually want an optimized format that only contains the information needed by your game, preferably in a format that can be loaded and used as-is without or with very minimal further processing.
Can you name file formats that match this "optimized format" description?
One more question. Let's say I have this one big .dat file in my game directory where data is present sequentially, and I load it.
Will I need to have tons of "#define X_ASSET_OFFSET" (and perhaps X_ASSET_SIZE) and then call ifstream::seekg() when I load every model?
