Jump to content
  • Advertisement
Sign in to follow this  
HellRiZZer

PAK File Flexibility

This topic is 4773 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hello everyone, Lately I've been working on a custom PAK file format, and I've stumbled across the problem. What I'm planning to do is to make PAK file as flexible as it's possible. I have a tEntryHeader structure like:
struct tEntryHeader
{
	enum
	{
		Package_Type_File,
		Package_Type_Directory
	};

	UINT m_uiDataOffset;		// Data offset in global file
	UINT m_uiDataSize;			// Data size (in bytes)
	UINT m_uiCompressionType;	// Compression type (reserved)
	UINT m_uiEntryType;			// 0 for Directory, 1 for a File
	UINT m_uiNumSubEntries;		// Number of sub-entries for this directory


	tEntryHeader()
	{
		m_uiDataOffset = 
		m_uiDataSize = 
		m_uiCompressionType = 
		m_uiEntryType = 
		m_uiNumSubEntries = 0;
	};

	tEntryHeader(UINT uiEntryType, UINT uiDataSize, UINT uiDataOffset = 0, UINT uiCompressionType = 0, UINT uiSubEntries = 0)
	{
		m_uiEntryType = uiEntryType;
		m_uiDataSize = uiDataSize;
		m_uiDataOffset = uiDataOffset;
		m_uiCompressionType = uiCompressionType;
		m_uiNumSubEntries = uiSubEntries;
	};

	virtual ~tEntryHeader()
	{

	};
};


and also a CPackageEntry class like:
class CPackageEntry : public CBinaryFile
{
protected:
	CPackageEntry();
public:
	CPackageEntry(tEntryHeader *pHeader,  CPackageEntry *pParent, void *pvData = NULL);
	
	enum
	{
		Package_IO_Default				= 0xF,
		Package_IO_Header_Incomplete,
		Package_IO_Header_Complete,
		Package_IO_Data_Incomplete,
		Package_IO_Data_Complete
	};

	virtual ~CPackageEntry();

	int AddEntry(CPackageEntry *pEntry);
	CPackageEntry *GetEntry(UINT uiIndex);
	int RemoveEntry(CPackageEntry *pEntry);

	virtual void Erase();
	virtual void Clear();
	virtual void Destroy();
	
	void SetData(BYTE *ucpData, UINT uiSize);
	BYTE *GetData();

	void SetName(const char *strName);
	char *GetName();

	void SetPackageType(UINT uiType);
	UINT GetPackageType();

	UINT GetCount();
	UINT GetTotalCount();

	UINT GetSize();
	UINT GetTotalSize();
	
	virtual int WriteHeader(void *pvFileHandle, UINT uiBytesToStream, long lOffset, UINT uiRefLo, bool bRecur = false);
	virtual int ReadHeader(void *pvFileHandle, UINT uiBytesToStream, long lOffset, UINT uiRefLoc, bool bRecur = false);

	virtual int WriteData(void *pvFileHandle, UINT uiBytesToStream, long lOffset, UINT uiRefLoc, bool bRecur = false);
	virtual int ReadData(void *pvFileHandle, UINT uiBytesToStream, long lOffset, UINT uiRefLoc, bool bRecur = false);

	tEntryHeader *GetHeader();

	void PrepareHeaders(tEntryHeader *pParentHeader);

	CPackageEntry *GetParent();
	CPackageEntry *GetTopParent();
	
protected:

	UINT				m_uiPackageState;
	UINT				m_uiPackageOffset;
	tEntryHeader		*m_pHeader;

private:
	CDoubleList<CPackageEntry *> m_lstSubEntries;

	CPackageEntry		*m_pParent;
	// Data type
	BYTE				*m_ucpData;
	
	UINT				m_uiPackageType;
};



What I'm having problems with is giving users enough flexibility in CPackage class
class CPackage : public CPackageEntry
{
public:
	CPackage();
	virtual ~CPackage();

	virtual int Write(void *pvFileHandle, void *pvData, UINT uiDataSize, UINT uiBytesToStream, long lOffset = -1, UINT uiRefLoc = File_Offset_Start);
	virtual int Read(void *pvFileHandle, void *pvData, UINT uiDataSize, UINT uiBytesToStream, long lOffset = -1, UINT uiRefLoc = File_Offset_Start);

};


to write headers/data where they want and how they want. For example, if you want a custom PAK file format (what I'm planning on using), what you do is you have the following structure:

Package
{
Package Header
Package Entry #1 Data
Package Entry #2 Data
Package Entry #3 Data
...
Package Entry #n Data
Package Entry Header #1
Package Entry Header #2
Package Entry Header #3
...
Package Entry Header #n
}


Or you may have the classic structure like this:
Package
{
Package Header
Package Entry Header #1
Package Entry Header #2
Package Entry Header #3
...
Package Entry Header #n
Package Entry #1 Data
Package Entry #2 Data
Package Entry #3 Data
...
Package Entry #n Data
}

So, I'm having problems with defining proper functions that will allow the user to emply ANY of those two formats (there may be more, it's up to the user to choose the format they want to write PAK file in) Can you people help me with this please? I can use existing Read/Write functions of CPackageEntry class to write both of those formats, but it looks ugly and not flexible enough. I also have to manage state of different returns to determine what to write next. Thanks for your help. [/source]

Share this post


Link to post
Share on other sites
Advertisement
just asking,why would you place yourdata infront of the headers?
do you gain a certain advantage of doing it that way?
otherwise

i would do it like that because you always read which files it contains and then open a file if it is contained within the pakfile

pakfile
{
pakfileheader
fileentry#1(with relative path information)
...
fileentry#n
filedata#1
...
filedata#n
}

Share this post


Link to post
Share on other sites
Well, if you read an article here, at GameDev, about Custom PAK File Format, in the thread discussing it they were talking about putting main header first, then all data and then the rest of the headers so that if you add a new entry, you can only write to the end of all data and re-write only that extra data and the headers - which are, compared to the total data size - are really, really small - thus, taking very little time to update an archive with newly added file.

But that's not the main point. The question still stands.

Share this post


Link to post
Share on other sites
one thing i could suggest is

to seperate the format and the package

class CPackage
{
void readerpackageheader();
CReader *m_pReader;
}


and now derive a reader/writer class from CReader

the reader write class handles the format issues and the CPackage you manages which reader/write to use

this way you can easily plug in another format


if this didn t answer your question i don t understand you problem

Share this post


Link to post
Share on other sites
I've already have that done. The CPackageEntry is derived from CBinaryFile, which in turn is derived from CBaseFile. I also have CTextFile deriven from CBaseFile, so it's just a matter of deriving CPackageEntry from CTextFile to write everything in text instead of binay format.
My question yet was, how/with what functions I can give the user that flexibility using which he can write PAK file in any way he can. E.g something simple yet powerful enough. I have an approximate version of it, but it's a little messy. Here's what I was thinking of doing:

I have enum like:
[SOURCE]
enum
{
Package_IO_Default = 0xF,
Package_IO_Header_Incomplete,
Package_IO_Header_Complete,
Package_IO_Data_Incomplete,
Package_IO_Data_Complete
};



When the user calls WriteHeader WITHOUT recursive request, the function will return the Package_IO_Header_Complete when it has done writing the callee' header to the file:

virtual int WriteHeader(void *pvFileHandle, UINT uiBytesToStream, long lOffset, UINT uiRefLo, bool bRecur = false);
virtual int ReadHeader(void *pvFileHandle, UINT uiBytesToStream, long lOffset, UINT uiRefLoc, bool bRecur = false);

virtual int WriteData(void *pvFileHandle, UINT uiBytesToStream, long lOffset, UINT uiRefLoc, bool bRecur = false);
virtual int ReadData(void *pvFileHandle, UINT uiBytesToStream, long lOffset, UINT uiRefLoc, bool bRecur = false);

But if the user calls WriteHeader WITH recursive request, it will go through the whole tree and write EVERY header to the file and only return Package_IO_Header_Complete when ALL headers have been written. Otherwise it will return Package_IO_Header_Incomplete.



Alternative version.
Make all headers/data eacessible in a linear manner. Of course, that would require some minor modifications to the CPackageEntry having a counter for that purpose. An example of linear access in a tree is hard to imagine, but here's an example:


Root (Package)
[
Entry #0 GetEntry(0)->GetEntry(0)->GetEntry(0) (if last one is not NULL)
Entry #1 GetEntry(0)->GetEntry(0)->GetEntry(1) (if last one is not NULL, otherwise it will progress upward on the tree)

...
And so on
]



Here's two examples of a decent, flexible designs. I'm not sure which one to go with, adn also my question was if you can think of something better.

Thanks.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!