Jump to content
  • Advertisement
Sign in to follow this  
Pilpel

How do game engineers pack their data?

This topic is 1319 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

In order to import 3d models, a 3d model file (i.e. COLLADA) and some maps are needed.

Having all that data out there in some folder isn't too smart because it can be viewed or even worse, edited, easily by anyone, so I noticed that many games have their data packed in one or many large files.

How do game engineers do that? Any thoughts on this matter?

Share this post


Link to post
Share on other sites
Advertisement

In general, people will tinker with and change the assets for your game if they love it.

So, even if you pack and compress or align your game data, consider making a small sdk with the tools so that people don't have to be ingenious to get at it. :)

Those tools includes pack readers, model viewers, etc.

Share this post


Link to post
Share on other sites

This is a nice article on prepping data offline ready for efficient loading: http://www.gamasutra.com/view/feature/132376/delicious_data_baking.php?page=1

 

Another thing to think about is whether you want to use a 'push' or a 'pull' approach to loading resources.

 

The pull approach is perhaps more natural, you have a bunch of resources set out into a logical (to a human) folder structure (possibly in some sort of zip-esque archive format), then your game code requests the assets it needs in some arbitrary order and they get loaded. Sometimes when you're loading a model, it might contain dependencies on some texture or sound, so you fire off loads for them too.

 

The problem with the pull approach is that you're seeking all over the place. Which is fine for solid state storage (SSD, cartridges, mobile) is OK for hard disk drives, but is a total nightmare for DVD/Blu-Ray, etc, as the seek costs can be huge.

 

The push model is an approach where you create a single file for a collection of stuff that will get loaded together. So if your game is level based and non-streaming, your tools might construct a single level file containing all the appropriate models, sounds and textures. The game code will request to load the level file, and the load process will push each of the resources it comes across into the right subsystem.

 

I guess with the current crop of platforms, and the fact that games stream so much dynamically, the push vs pull choice is less important, and it's always possible to make the pull model work OK by carefully laying out resources on the storage media, but I think the push approach is the better one if you're planning out a system from scratch.

Share this post


Link to post
Share on other sites

If you're using C or c++, look into physicsfs.

 

It allows loading resources from compressed archive formats such as zip, 7z, and wad. It acts like a filesystem so you can stream files from it or just load them whole into ram. Definitely worth the effort to integrate. 

Share this post


Link to post
Share on other sites

Firstly, you probably don't want an interchange format like COLLADA or the native format of your modelling package in your final game; it has all the data needed for editing your model, and a lot of that simply isn't needed in the final game, so you'll usually want an optimized format that only contains the information needed by your game, preferably in a format that can be loaded and used as-is without or with very minimal further processing.

 

Can you name file formats that match this "optimized format" description?

 

One more question. Let's say I have this one big .dat file in my game directory where data is present sequentially, and I load it.

Will I need to have tons of "#define X_ASSET_OFFSET" (and perhaps X_ASSET_SIZE) and then call ifstream::seekg() when I load every model?

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!