Jump to content

  • Log In with Google      Sign In   
  • Create Account

Asset file format


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
5 replies to this topic

#1 Alundra   Members   -  Reputation: 867

Like
0Likes
Like

Posted 29 June 2014 - 05:38 PM

Hi all,

I have differents file format : mesh, texture, animation, and surely soon material.

Actually I have differents extension but I would have only one file format ".asset".

My idea is to have a class Asset who has load/save and memory file data.

At the end the asset is just two uint32 who is the type of the asset and the file data size then the file data.

Each class (mesh, texture, animation, material) has function to write to memory and read from memory.

Is it a bad way to handle that and should be done in another way ?

Thanks for the help


Edited by Alundra, 29 June 2014 - 05:42 PM.


Sponsor:

#2 SeanMiddleditch   Members   -  Reputation: 5851

Like
2Likes
Like

Posted 30 June 2014 - 12:45 AM

There's nothing particularly wrong with this. It can be beneficial to have separate file types with different extensions, but it's not critically important in any way.

It's easier to write tools that open the files that do some kind of asset-type-specific processing. For instance, a custom image viewer for your texture type, or a model previewer. Including a type marker in the file itself works, but it makes it a bit harder on some OSes.

The format itself that you propose is actually just fine. The one thing I would add is that you want to make the on-disk format and the in-memory format as similar as possible so as to avoid any kind of data transformation or processing necessary when loading the file. This often means choosing data formats that use arrays indices rather than pointers internally, since pointers can't be directly saved to disk. You'll likely still want a data validation step that at least verifies indices are in valid ranges and won't be a gaping security hole, but iterating over a bunch of read-only data and verifying it is much faster than modifying it (and can be parallelized fairly easily).

#3 haegarr   Crossbones+   -  Reputation: 4312

Like
1Likes
Like

Posted 30 June 2014 - 01:18 AM

Usually there are 2, perhaps 3 situations to be distinguished:

(1) "Raw assets" as are loaded by an editor for the purposes of being viewed and perhaps integrated into the project,

(2) "runtime assets" that are loaded by the game engine,

(3) and perhaps "runtime assets" that are loaded by the game engine, either provided for hot-swapping, software updates, or modding.

 

Raw assets can be stored in the native file format of the DCC or in a common interchange file format (e.g. Collada for meshes), obviously to have a chance to re-load it into the DCC and make changes. These assets are further given by individual files, although a single file may provide more than one asset.

 

Runtime assets are usually provided in a game engine specific file format. The purpose is to yield in fast loading, which means a binary format because it is more compact but also requires less pre-processing in the game engine. The tool-chain is responsible to convert from raw assets to runtime assets.

 

With respect to the normal game play runtime, one don't want to have to open one file per asset. This is because file opening costs time, and having many more or less small files consumes more storage footprint. The solution is to use some kind of package file format. It further allows assets to be sequenced which in turn allows for shorter load times.

 

Now, with respect to hot-swapping, updates, and/or modding, storing assets in a single big package is bad, because replacing an asset in a package is often painful. A solution is to use the same file format but allowing for single, additional files so that the content of these files override the corresponding content of the regular packages. The runtime asset loader can handle this for example if the overriding asset files are stored in a dedicated file directory apart from the main asset files.

 

With packages the question of different file suffixes for runtime assets vanishes obviously. Further, it it easier for the runtime asset loader to handle a single file suffix. It has to investigate the file content anyway. So for me using a single suffix for runtime assets is a good way.

 

Just my 2 Cents.



#4 Alundra   Members   -  Reputation: 867

Like
0Likes
Like

Posted 30 June 2014 - 08:05 AM


Runtime assets are usually provided in a game engine specific file format. The purpose is to yield in fast loading, which means a binary format because it is more compact but also requires less pre-processing in the game engine. The tool-chain is responsible to convert from raw assets to runtime assets.

I'm using that, each class has his file format and asset class is just an header at the end, and one function to just read the header can be added to check the asset type.

I have a PakFile class who uses ZLib for compression, I read from memory from this PakFile, I have an entry table.


Edited by Alundra, 30 June 2014 - 08:06 AM.


#5 mynameisnafe   Members   -  Reputation: 252

Like
0Likes
Like

Posted 01 July 2014 - 12:48 PM

I only throw my tuppence in here as I'm currently undertaking a similar task:

Assimp does this.. a 'aiScene' is loaded which contains meshes, materials, animations - materials contain raw bits of textures, and colour information.  Assimp does a fairly good job of this for many formats - in fact I came this ' ' close to ripping of their code for my dissertation*

Project 1 (An 'editor'):

 - I'm using Assimp to load models with all their materials into MyMesh class, then writing out MyMesh to a binary file optimized for reading into memory quickly (think memcpy over for(i = 0;...) when loading this file)
- include geometry, anim, materials etc in the one file.

Project 2 (A 'game engine'/renderer):
- Load the 'optimized' custom format, use it.

 

On the other hand, there are some compressed binary model file formats out there - I've heard good things about OpenCTM, for example. (Google if curious).

*of course I mean properly referencing, and simply using Assimp instead of writing my own. If anyone can find documentation for the animation chunks in the .3ds file I will give them cookies.
 



#6 Alundra   Members   -  Reputation: 867

Like
0Likes
Like

Posted 01 July 2014 - 02:18 PM

I would recommend you to not have material in mesh class but only MaterialID who is index of material in subset.

Then you can have a MeshComponent who has a mesh pointer (from manager) and array of material (from manager too).






Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS