Jump to content

  • Log In with Google      Sign In   
  • Create Account


Size of binary file? [C++]


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
10 replies to this topic

#1 MatsK   Members   -  Reputation: 223

Like
0Likes
Like

Posted 27 February 2011 - 01:25 PM

Hi!
I'm trying to find the size of a binary file.
I've Googled around, and I constantly seem to be coming up with answers that are very similar to the code I have now;

ifstream CurrentFile(Filename, ios::out | ios::in| ios::binary);
			CurrentFile.seekg(ios::ate);
			ifstream::pos_type size = CurrentFile.tellg();

			char *MemBlock = new char[size];
			CurrentFile.seekg(0, ios::beg);
			CurrentFile.read(MemBlock, size);


Is this correct? Somehow, I don't think it is, because my program is always giving me an allocation error when I'm trying to initialize the MemBlock array.
By the way, what does ATE stand for?

Sponsor:

#2 PaladinOfKaos   Members   -  Reputation: 134

Like
0Likes
Like

Posted 27 February 2011 - 01:32 PM

Yes, that's wrong. It should be one of too things:
// option A
ifstream CurrentFile(Filename, ios::in | ios::binary);
CurrentFile.seekg(0, ios::end);
ifstream::pos_type size = CurrentFile.tellg();

// option B
ifstream CurrentFile(Filename, ios::in | ios::binary | ios::ate);
ifstream::pos_type size = CurrentFile.tellg();

I got rid of your ios::out flags, since you really shouldn't set them on an ifstream

As for what ios::ate stands for, it's probably the past-tense of "eat".

#3 Buckeye   Crossbones+   -  Reputation: 3804

Like
0Likes
Like

Posted 27 February 2011 - 01:32 PM

Use ios::end and check for a 0 filesize.

This link describes the appropriate arguments for tellg.

It may not make a difference for your app, but, in general, if you're reading binary data, use an unsigned buffer. I.e., unsigned char* rather than char*.

Good programming practice includes error-chiecking - such as whether the file was successfully opened, 0 file size, etc.

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.


#4 nobodynews   Crossbones+   -  Reputation: 1764

Like
0Likes
Like

Posted 27 February 2011 - 01:55 PM

As for what ios::ate stands for, it's probably the past-tense of "eat".

I couldn't find anything official but I think it is for at end (ATEnd). A search ( http://www.google.com/search?hl=&q=ios%3A%3Aate+%22at+end%22&sourceid=navclient-ff&rlz=1B3GGLL_enUS410US410&ie=UTF-8#sclient=psy&hl=en&safe=off&rlz=1B3GGLL_enUS410US410&q=ios:%3Aate+%22at+end%22&aq=f&aqi=&aql=&oq=&pbx=1&bav=on.1,or.&fp=18e4c0cc530c3619 ) suggests this is a common belief but common doesn't mean correct.

C++: A Dialog | C++0x Features: Part1 (lambdas, auto, static_assert) , Part 2 (rvalue references) , Part 3 (decltype) | Write Games | Fix Your Timestep!


#5 MatsK   Members   -  Reputation: 223

Like
0Likes
Like

Posted 27 February 2011 - 02:38 PM

Hmph. I still get an allocation error.
After checking, it turns out that I'm trying to allocate 3,9 gigabytes of memory. I have 6gigs of memory on my PC, but considering that Windows 7 is using close to a gig (?), I don't think there's enough space.
What worries me is that the file I'm reading is 1kb...

#6 MatsK   Members   -  Reputation: 223

Like
0Likes
Like

Posted 27 February 2011 - 03:18 PM

Nevermind, I figured it out!
Thanks guys!

#7 kbw   Members   -  Reputation: 106

Like
0Likes
Like

Posted 27 February 2011 - 03:24 PM

Use stat() to find the size of a file. It accesses the file's metadata rather than opening the file itself, so is more efficient.

#8 Buckeye   Crossbones+   -  Reputation: 3804

Like
0Likes
Like

Posted 27 February 2011 - 04:00 PM

Nevermind, I figured it out!
Thanks guys!

Part of the idea here on gamedev is sharing solutions to problems. So, what was the problem and what was the fix (in case someone else runs into a similar problem)?

Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.


#9 rip-off   Moderators   -  Reputation: 7660

Like
0Likes
Like

Posted 27 February 2011 - 05:33 PM

Allocating such a large block could always fail due to fragmentation, especially if your processes are 32bit.

#10 MatsK   Members   -  Reputation: 223

Like
1Likes
Like

Posted 28 February 2011 - 03:52 AM


Nevermind, I figured it out!
Thanks guys!

Part of the idea here on gamedev is sharing solutions to problems. So, what was the problem and what was the fix (in case someone else runs into a similar problem)?


The problem was that the filename was wrong. When I tried to print it out to the console, it only displayed part of the filename. That probably had something to do with the way I was doing it though.
The thing is, my program converts files from *.bmp to *.png, and I had already deleted all my extracted *.bmp files when this particular piece of code was being run, so the filename was not found.
I ended up converting the char* to a std::string and replacing 'bmp' with 'png'.

char* Filename = new char[m_Entries[i].FilenameLength];
			memcpy(Filename, &m_Entries[i].Filename, m_Entries[i].FilenameLength);
			string StrFilename(Filename, Filename + m_Entries[i].FilenameLength);
			StrFilename = Replace(StrFilename, "bmp", "png");

			ifstream CurrentFile(StrFilename.c_str(), ios::in | ios::binary | ios::ate);
			//CurrentFile.seekg(0, ios::end);
			ifstream::pos_type size = CurrentFile.tellg();


string Replace(string Str, string From, string To)
	{
		int Position = Str.find(From);

		while(Position != string::npos)
		{
			Str = Str.replace(Position, From.length(), To);
			Position = Str.find(From, Position + From.length());
		}

		return Str;
	}


I had already written the Replace() function before I ran into this problem.
Now that I think about it, I should probably replace 'bmp' with '.bmp' and 'png' with '.png' in order to minimize the chance that a random character sequence that is actually part of the filename itself is replaced.

Here is the full sourcecode for the class I'm working on:

#include "stdafx.h"
#include "FileArchive.h"
#include <assert.h>
#include <iostream>
#include "Compress.h"
#include "FreeImage.h"

namespace Archives
{
	string Replace(string Str, string From, string To);

	FileArchive::FileArchive(string Path)
	{
		char Signature[8];
		char Version[4];
		char ManifestOffset[4];
		char NumFiles[4];

		m_ArchivePath = Path;

		ifstream Archive(Path.c_str(), ios::in | ios::binary);
		Archive.read(Signature, sizeof(FARSignature));

		string StrSignature(Signature, Signature + 8);

		assert(StrSignature == "FAR!byAZ");
		cout << "Signature: " << StrSignature << "\r\n";
		
		Archive.read(Version, 4);
		m_Version = *((int*)Version);

		assert(m_Version == 1 || 3);

		Archive.read(ManifestOffset, 4);
		m_ManifestOffset = *((unsigned int*)ManifestOffset);
		
		Archive.seekg(m_ManifestOffset, ios::beg);

		Archive.read(NumFiles, 4);
		m_NumFiles = *((unsigned int*)NumFiles);

		Archive.close();
	}

	void FileArchive::Process()
	{
		ifstream Archive(m_ArchivePath.c_str(), ios::in | ios::binary);

		char DecompressedDataSize[4];
		char CompressedDataSize[3];
		char DataType[1];
		char DataOffset[4];
		char Compressed[1];
		char AccessNumber[1];
		char FilenameLength[2];
		char TypeID[4];
		char FileID[4];
		char* Filename;
		
		//Seek past the number of files...
		Archive.seekg(m_ManifestOffset + 4, ios::beg);

		for(unsigned int i = 0; i < m_NumFiles; i++)
		{
			FAR3Entry Entry;
			
			Archive.read(DecompressedDataSize, sizeof(unsigned int));
			Entry.DecompressedDataSize = *((unsigned int*)DecompressedDataSize);

			Archive.read(CompressedDataSize, 3);
			Entry.CompressedDataSize = *((unsigned int*)CompressedDataSize);

			Archive.read(DataType, sizeof(unsigned char));
			Entry.DataType = *((unsigned char*)DataType);

			Archive.read(DataOffset, sizeof(unsigned int));
			Entry.DataOffset = *((unsigned int*)DataOffset);

			Archive.read(Compressed, sizeof(unsigned char));
			Entry.Compressed = *((unsigned char*)Compressed);

			Archive.read(AccessNumber, sizeof(unsigned char));
			Entry.AccessNumber = *((unsigned char*)AccessNumber);

			Archive.read(FilenameLength, sizeof(unsigned short));
			Entry.FilenameLength = *((unsigned short*)FilenameLength);

			Archive.read(TypeID, sizeof(unsigned int));
			Entry.TypeID = *((unsigned int*)TypeID);

			Archive.read(FileID, sizeof(unsigned int));
			Entry.FileID = *((unsigned int*)FileID);

			Filename = new char[Entry.FilenameLength];
			Archive.read(Filename, Entry.FilenameLength);
			memcpy(&Entry.Filename, Filename, Entry.FilenameLength);

			string StrFilename(Filename, Filename + Entry.FilenameLength);
			cout << "Filename: " << StrFilename << "\r\n";

			m_Entries.push_back(Entry);

			char* Buffer = new char[Entry.DecompressedDataSize];
			
			//Don't convert tgas...
			if(StrFilename.find(".tga") == string::npos)
			{
				FARExtractItemFromFileByName(m_ArchivePath.c_str(), StrFilename.c_str(), 
					(unsigned char**)&Buffer, 1);

				FREE_IMAGE_FORMAT Fif = FreeImage_GetFIFFromFilename(StrFilename.c_str());
				
				if(Fif != FIF_UNKNOWN)
				{
					FIBITMAP *Img = FreeImage_Load(Fif, StrFilename.c_str());
					FreeImage_Save(FREE_IMAGE_FORMAT::FIF_PNG, Img, Replace(StrFilename, ".bmp", ".png").c_str());

					remove(StrFilename.c_str());
					//delete Img; //This causes an assertion... o_O
				}
			}

			delete Buffer;
		}

		RecreateArchive();
	}

	void FileArchive::RecreateArchive()
	{
		int ManifestOffset = 11;

		ofstream Archive(Replace(m_ArchivePath, ".dat", ".tmp").c_str());
		Archive << "FAR!byAZ";	//Signature
		Archive << 3;			//Version
		Archive << 0x00;		//ManifestOffset

		for(unsigned int i = 0; i < m_NumFiles; i++)
		{
			char* Filename = new char[m_Entries[i].FilenameLength];
			memcpy(Filename, &m_Entries[i].Filename, m_Entries[i].FilenameLength);
			string StrFilename(Filename, Filename + m_Entries[i].FilenameLength);
			StrFilename = Replace(StrFilename, ".bmp", ".png");

			ifstream CurrentFile(StrFilename.c_str(), ios::in | ios::binary | ios::ate);
			//CurrentFile.seekg(0, ios::end);
			ifstream::pos_type size = CurrentFile.tellg();

			unsigned char *MemBlock = new unsigned char[size];
			CurrentFile.seekg(0, ios::beg);
			CurrentFile.read((char*)MemBlock, size);

			unsigned char* Dst = mynew<unsigned char>((int)size - 1);
			unsigned char* DstEnd = compress((unsigned char*)MemBlock, (unsigned char*)MemBlock + size, 
				Dst, Dst + size - 1, false);
			
			cout << "Bob the builder, can he compress it? \r\n"; 

			//Write the compressed...
			if(DstEnd)
			{
				unsigned int DstLength = DstEnd - Dst;

				//TODO: Write FAR3 Compression header.
				cout << "Yes he can!\r\n\r\n";
				
				//Update the offset for the location of the data!
				m_Entries[i].DataOffset = Archive.tellp();

				for(unsigned int i = 0; i < DstLength; i++)
					Archive << Dst[i];
			}
			else //... or uncompressed file.
			{
				cout << "No he can't!\r\n\r\n";
				
				m_Entries[i].Compressed = 0x00;

				for(int i = 0; i < size; i++)
					Archive << Dst[i];
			}
			
			delete Dst;
			delete MemBlock;
			delete Filename;
			CurrentFile.close();
		}

		ifstream::pos_type ManifestOffsetStart = Archive.tellp();

		for(unsigned int i = 0; i < m_NumFiles; i++)
		{
			Archive << m_Entries[i].DecompressedDataSize;
			//TODO: Write this as only 3 bytes!
			Archive << m_Entries[i].CompressedDataSize;
			Archive << m_Entries[i].DataType;
			Archive << m_Entries[i].DataOffset;
			Archive << m_Entries[i].Compressed;
			Archive << m_Entries[i].AccessNumber;
			Archive << m_Entries[i].FilenameLength;
			Archive << m_Entries[i].TypeID;
			Archive << m_Entries[i].FileID;
			
			//Note: Not entirely sure if this is the correct way to get the filename down...
			for(unsigned int j = 0; j < m_Entries[i].FilenameLength; j++)
				m_Entries[i].Filename[j];
		}
	}

	string Replace(string Str, string From, string To)
	{
		int Position = Str.find(From);

		while(Position != string::npos)
		{
			Str = Str.replace(Position, From.length(), To);
			Position = Str.find(From, Position + From.length());
		}

		return Str;
	}}


I'm using FreeImage to convert the images, and some black voodoo magic to recompress an archive.
Oh, and, in case you can't figure it out by looking at this code, here's what a FAR3 entry looks like:

struct FAR3Entry {
	unsigned int DecompressedDataSize;
	unsigned int CompressedDataSize; /* Only 3 bytes large; refers to the total data's size including the RefPack header */
	unsigned char DataType; /* Normally equals 0x80 to denote that the data is in a RefPack container */
	unsigned int DataOffset; /* Relative to the beginning of the FAR file */
	unsigned char Compressed; /* Normally equals 0x01 */
	unsigned char AccessNumber; /* Refers to the number of times that the data at the specified offset has been used for other entries: Normally equals 0x00 (and works its way up) */
	unsigned short FilenameLength; /* Note that the file name does not terminate with a null character */
	unsigned int TypeID;
	unsigned int FileID;
	char Filename[256];
};

Lesson learned?
If you're trying to find the size of a 1,5kb file and end up allocating 3,9gigs for it, the file probably wasn't opened, or you did something wrong.

#11 MatsK   Members   -  Reputation: 223

Like
0Likes
Like

Posted 28 February 2011 - 04:08 AM

Use stat() to find the size of a file. It accesses the file's metadata rather than opening the file itself, so is more efficient.


Moot point, since I need to read the file anyways. See above code.




Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS