packing assets

Started by
2 comments, last by KulSeran 13 years, 2 months ago
hi,

we've been developing on an implementation of the collada xml schema and we're currently processing a few exported (blender) files, and our pipeline throws out a bunch of files, one for each Model.

Now, on loading up a certain level/map or whatever, theres a class describing the needed assets, and posiotions/modifications that need to be made to loaded assets. When a level/map or whatever is loaded the resourcemanager needs to open up each file and load it.

All this file opening is slow so we're searching a solution to pack up things, maybe with compression to increase loading speed. We only want to have one or only few files (filecontainers), which would mean to only need to open one file, and then only load/extreact/unpack the required assets from it.

to excerpt all this, we're searching for a free/opensource lib that can pack all assets into one zip/7zip or any other compressed archive, so we only have to open that archive and can pick the assets we want to extract from it. (maybe also have our sound files, images, movies, packed up there)


furthermore I want to ask if this approach is somehow reasonable, or if it's just plain stupid
Advertisement
You might want to check out the zlib-library.
It can be found at http://www.zlib.net/,

There is tons of information/tutorials about how to use the library to extract data from a compressed zip-file.
You should also be able to find quite a few C++ wrappers that makes extracting so much easier in my own opinion.

As for packing the files, just use any software that is able to create zip-files and add files to the archive.

You should also be able to find quite a few C++ wrappers that makes extracting so much easier in my own opinion.


i currently can't imagine for what i could need a wrapper? please explain


and what common free packing libraries are there? i guess this is not only a trivial decision for game assets, because packers vary in compression quality, and the time needed to decompress.

are there any guidlines on strong a file should be packed, so the computing time unpacking the file of such format would fit best together with the io reading time

and what common free packing libraries are there? i guess this is not only a trivial decision for game assets, because packers vary in compression quality, and the time needed to decompress.
are there any guidlines on strong a file should be packed, so the computing time unpacking the file of such format would fit best together with the io reading time

You could look into PhysFS. It will wrap over simple formats like Zip files, and Quake 1 Pak files.

You should be less concerned with the decompression speed ( zlib is super fast ), and more concerned with removing the processing after load. You should, in theory, be able to load up one big compressed file into memory, read the header, and then walk through the compressed file setting up pointers, and be done without calling a single malloc/new. Eliminate text format files and anything that needs pre-processing to be useful. That doesn't mean you can't store text files for stuff, just make a packing utility to convert to a pre-processed binary format before your game uses it.

This topic is closed to new replies.

Advertisement