• Advertisement
Sign in to follow this  

Asset file format

This topic is 1301 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi all,

I have differents file format : mesh, texture, animation, and surely soon material.

Actually I have differents extension but I would have only one file format ".asset".

My idea is to have a class Asset who has load/save and memory file data.

At the end the asset is just two uint32 who is the type of the asset and the file data size then the file data.

Each class (mesh, texture, animation, material) has function to write to memory and read from memory.

Is it a bad way to handle that and should be done in another way ?

Thanks for the help

Edited by Alundra

Share this post


Link to post
Share on other sites
Advertisement
There's nothing particularly wrong with this. It can be beneficial to have separate file types with different extensions, but it's not critically important in any way.

It's easier to write tools that open the files that do some kind of asset-type-specific processing. For instance, a custom image viewer for your texture type, or a model previewer. Including a type marker in the file itself works, but it makes it a bit harder on some OSes.

The format itself that you propose is actually just fine. The one thing I would add is that you want to make the on-disk format and the in-memory format as similar as possible so as to avoid any kind of data transformation or processing necessary when loading the file. This often means choosing data formats that use arrays indices rather than pointers internally, since pointers can't be directly saved to disk. You'll likely still want a data validation step that at least verifies indices are in valid ranges and won't be a gaping security hole, but iterating over a bunch of read-only data and verifying it is much faster than modifying it (and can be parallelized fairly easily).

Share this post


Link to post
Share on other sites

Usually there are 2, perhaps 3 situations to be distinguished:

(1) "Raw assets" as are loaded by an editor for the purposes of being viewed and perhaps integrated into the project,

(2) "runtime assets" that are loaded by the game engine,

(3) and perhaps "runtime assets" that are loaded by the game engine, either provided for hot-swapping, software updates, or modding.

 

Raw assets can be stored in the native file format of the DCC or in a common interchange file format (e.g. Collada for meshes), obviously to have a chance to re-load it into the DCC and make changes. These assets are further given by individual files, although a single file may provide more than one asset.

 

Runtime assets are usually provided in a game engine specific file format. The purpose is to yield in fast loading, which means a binary format because it is more compact but also requires less pre-processing in the game engine. The tool-chain is responsible to convert from raw assets to runtime assets.

 

With respect to the normal game play runtime, one don't want to have to open one file per asset. This is because file opening costs time, and having many more or less small files consumes more storage footprint. The solution is to use some kind of package file format. It further allows assets to be sequenced which in turn allows for shorter load times.

 

Now, with respect to hot-swapping, updates, and/or modding, storing assets in a single big package is bad, because replacing an asset in a package is often painful. A solution is to use the same file format but allowing for single, additional files so that the content of these files override the corresponding content of the regular packages. The runtime asset loader can handle this for example if the overriding asset files are stored in a dedicated file directory apart from the main asset files.

 

With packages the question of different file suffixes for runtime assets vanishes obviously. Further, it it easier for the runtime asset loader to handle a single file suffix. It has to investigate the file content anyway. So for me using a single suffix for runtime assets is a good way.

 

Just my 2 Cents.

Share this post


Link to post
Share on other sites

Runtime assets are usually provided in a game engine specific file format. The purpose is to yield in fast loading, which means a binary format because it is more compact but also requires less pre-processing in the game engine. The tool-chain is responsible to convert from raw assets to runtime assets.

I'm using that, each class has his file format and asset class is just an header at the end, and one function to just read the header can be added to check the asset type.

I have a PakFile class who uses ZLib for compression, I read from memory from this PakFile, I have an entry table.

Edited by Alundra

Share this post


Link to post
Share on other sites

I only throw my tuppence in here as I'm currently undertaking a similar task:

Assimp does this.. a 'aiScene' is loaded which contains meshes, materials, animations - materials contain raw bits of textures, and colour information.  Assimp does a fairly good job of this for many formats - in fact I came this ' ' close to ripping of their code for my dissertation*

Project 1 (An 'editor'):

 - I'm using Assimp to load models with all their materials into MyMesh class, then writing out MyMesh to a binary file optimized for reading into memory quickly (think memcpy over for(i = 0;...) when loading this file)
- include geometry, anim, materials etc in the one file.

Project 2 (A 'game engine'/renderer):
- Load the 'optimized' custom format, use it.

 

On the other hand, there are some compressed binary model file formats out there - I've heard good things about OpenCTM, for example. (Google if curious).

*of course I mean properly referencing, and simply using Assimp instead of writing my own. If anyone can find documentation for the animation chunks in the .3ds file I will give them cookies.
 

Share this post


Link to post
Share on other sites

I would recommend you to not have material in mesh class but only MaterialID who is index of material in subset.

Then you can have a MeshComponent who has a mesh pointer (from manager) and array of material (from manager too).

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement