Sign in to follow this  
miggs

packing assets

Recommended Posts

hi,

we've been developing on an implementation of the collada xml schema and we're currently processing a few exported (blender) files, and our pipeline throws out a bunch of files, one for each Model.

Now, on loading up a certain level/map or whatever, theres a class describing the needed assets, and posiotions/modifications that need to be made to loaded assets. When a level/map or whatever is loaded the resourcemanager needs to [b]open up each file[/b] and load it.

All this file opening is slow so we're searching a solution to pack up things, maybe with compression to increase loading speed. We only want to have one or only few files (filecontainers), which would mean to only need to [b]open one file[/b], and then only [b]load/extreact/unpack[/b] the required assets from it.

to [b]excerpt[/b] all this, we're searching for a [b]free/opensource lib [/b]that can [b]pack all assets[/b] into one [b]zip/7zip[/b] or any other [b]compressed archive[/b], so we only have to open that archive and can pick the assets we want to extract from it. (maybe also have our sound files, images, movies, packed up there)


furthermore I want to ask if this approach is somehow reasonable, or if it's just plain stupid

Share this post


Link to post
Share on other sites
You might want to check out the zlib-library.
It can be found at [url="http://www.zlib.net/"]http://www.zlib.net/[/url],

There is tons of information/tutorials about how to use the library to extract data from a compressed zip-file.
You should also be able to find quite a few C++ wrappers that makes extracting so much easier in my own opinion.

As for packing the files, just use any software that is able to create zip-files and add files to the archive.

Share this post


Link to post
Share on other sites
[quote name='r1nux' timestamp='1296727312' post='4768950']
You should also be able to find quite a few C++ wrappers that makes extracting so much easier in my own opinion.
[/quote]

i currently can't imagine for what i could need a wrapper? please explain


and what common free packing libraries are there? i guess this is not only a trivial decision for game assets, because packers vary in compression quality, and the time needed to decompress.

are there any guidlines on strong a file should be packed, so the [b]computing[/b] time unpacking the file of such format would fit best together with the io [b]reading[/b] time

Share this post


Link to post
Share on other sites
[quote name='miggs' timestamp='1296745551' post='4769031']
and what common free packing libraries are there? i guess this is not only a trivial decision for game assets, because packers vary in compression quality, and the time needed to decompress.
are there any guidlines on strong a file should be packed, so the [b]computing[/b] time unpacking the file of such format would fit best together with the io [b]reading[/b] time
[/quote]
You could look into [url="http://icculus.org/physfs/"]PhysFS[/url]. It will wrap over simple formats like Zip files, and Quake 1 Pak files.

You should be less concerned with the decompression speed ( zlib is super fast ), and more concerned with removing the processing after load. You should, in theory, be able to load up one big compressed file into memory, read the header, and then walk through the compressed file setting up pointers, and be done without calling a single malloc/new. Eliminate text format files and anything that needs pre-processing to be useful. That doesn't mean you can't store text files for stuff, just make a packing utility to convert to a pre-processed binary format before your game uses it.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this