Game achrive files

Started by
2 comments, last by sakky 19 years, 9 months ago
What do you think of this? This archive file consists of chunks of data. Each chunk of data may have its own directory list or chunk list. Each chunk of data has a unique identity and offset into the file where it lies. A copy of these identities and offset are kept towards the beginning of the file. Archive Header Structure
  • Special archive signature code
  • Archive version number
  • Number of entries in archive directory
  • Offset to directory chunk
Directory Entry Structure
  • Chunk identity code
  • Offset to chunk header
Chunk Header Structure
  • Chunk identity code
  • Chunk type
  • Size of chunk data
  • Offset to chunk data
This is something I just came up with in the last 10 minutes. I need lots of work but it looks promising. The archive file is similar to other archive files in the aspect that different data may be inserted or removed from the archive. The main difference, however, is that it supports many smaller archives with in the archive or inner-archives with the main archive.
Take back the internet with the most awsome browser around, FireFox
Advertisement
If you add a file to an existing archive, you must add an entry to the directory structure and thus move all archive chunks in the file. It's better to store the directory information at the end of the archive, 'cause then you would only have to shift the directory structure which is most probably a lot smaller. Adding files to archives is the most common modification of archives, so optimizing the format to that operation is preferred I guess?
I'll actually be employing a smiliar concept as this in my resource files, but done a slightly different way. First of all, all my resource files (be it a texture, a 3D model, or a level) will be stored in the same file structure. The same chunk identifications will be common between all the different files. The file itself isn't indexed, it's just sequential access though. Anyway, chunks in my files will either be grouping, in which case they contain no data by themselves, or data chunks. In the case of grouping chunks, the data for that chunk is all stored in "child" chunks that are contained inside that chunk. You can't compress these ones. Data chunks however can be compressed. I would probably only compress large data chunks that contain things like geometric data, or texture data, but any data chunk could be compressed.

This system means I can traverse the structure of the file without decompressing anything. If I was after something in perticular, I can determine what I actually want, then decompress just that portion. It also allows new chunks to be added or removed easily, because the structure is unaffected by the compression.

I have one general chunk header, which at this time I think will be something like this:
-Chunk ID			;(int)-Chunk size/compression flag	;(int)	Compression flag	;bit 32	Chunk size		;bits 1-31-Chunk version			;(int)


I can use the upper bit of the size member to indicate if compression is in use, because it's quite unlikely I'll be needing a 4GB chunk anytime soon. The only relevant part of my file structure is the chunk header. Each chunk then has a fixed structure that defines its data members. This format has the main limitation however that it does not allow for repetition of data. If for some reason I want the same data to be referanced twice, I need to store it twice. I'll think this over more the closer I get to implementation, and decide if it's enough of a limitation to warrant desiging a more sophisticated method. This is the simplest, so it's the most attractive from that front.

Oh, and I should mention. I'll be using an implementation of LZ77 compression, so it's fairly straightforward, and I don't need any special information to be stored for each compressed chunk, I just have to know it's compressed.
I thought of something that reminded me of a .WAD. I wanted one file that would contain the entire data for the game or application. I was thinking about making an archive framework. The framework would have a small hatchery that will contain the objects that are used in the file. So a call to the framework might look like this:


pFrameWorkParent->InitializeArchive( “SomeArchiveFile.ArchiveExtension” );
pFrameWorkParent->GetThisChunk( ID_SOMECHUNK, pChunkObject );

DWORD dwSize = pChunkObject->GetChunkSize( );
DWORD dwID = pChunkOBject->GetChunkID( );

pFrameWorkParent->GotToThisChunk( pChunkObject );

LPVOID pvData = ( BYTE* )pChunkObject->GetChunkData( );
pChunkObject->Release( );

pFrameWorkParent->UnInitilize( );
pFrameWorkParent->Release( );



Notice how I used the chunk object again to set the file read position. This is because, if a user requests a chunk (by identity) that does not exist, then they get nothing or NULL. If the chunk they requested does indeed exist in the file, then a pointer to a new chunk object is returned. In order for the user to make use of this chunk, they need to read data to and from it. I want to file input/output to be controlled by the parent ONLY (data abstraction). A chunk does not need to now what a file is. A chunk is just the interface to some data that reads and writes to it. Anyways, the parent object then resets the file pointer to the specified chunk’s location or offset. The parent archive will now initialize some data of the chunk with the size, type, and offset information about the chunk’s data.

Pretty simple huh? I though about implementing the framework in COM. If not come, then just some cool C++ classes. I do, however, want the archive framework to be consolidated into a single library (static or DLL). I think it would be cool to develop something like this that is fast and reliable. Then I could offer it to every one for there own use of it. Maybe I can offer a simple DLL to the framework as a temporary project replacement, and offer the source code in a DLL project so the use may perfect his or her own archive interface.

Sounds good, yes?
Take back the internet with the most awsome browser around, FireFox

This topic is closed to new replies.

Advertisement