Skip to main content
GameDev.net gamedev.net
🔒 Locked

[.net] Writing Binary Files in C#, Reading them in C++

Started by ArchG Mar 31, 2005 at 11:43 AM 17 replies 25.2k views
Original Post
ArchG
ArchG
Hello, i'm working on a small game in C++ right now, and I needed a MonsterObject Creator, so since C# seems faster and easier for such a thing, I decided to make the MonsterObject Creator with C#.. Right now I have a monster structure, like this.. 

struct MonsterStruct
{
string sName;
int ID;
int HP;
int Sight;
int Speed;
bool bFriendly;
float RespawnRate;
int ObjectDropped1;
float fValue1;
int ObjectDropped2;
float fValue2;
};
MonsterStruct[] Monster;
Monster = new MonsterStruct[255];
Im wondering what the best way is to write a bunch of structures like that to a binary file that can be read in C++..I figured a BinaryWriter was really the only way to go.. 

//Pretend Monster is a MonsterStruct array of 255 and is already created and filled
FileStream fs = new FileStream("Monsters.dat", FileMode.CreateNew);
BinaryWriter w = new BinaryWriter(fs);
//What would be the easest way to write it?
w.Write(??);
And Then a bonus...how would I read that in C++ (I guess it's hard to know how to read it until you know how your writing it though huh?)
Undergamer
Undergamer
I would create a method in your C# class which is called "byte[] Serialize()". Using the array classes already in C# you can append the bytes for each field of the class so that they match up with what would be read in C++. There are different datatypes and assuming that you store it in order the correct way it shouldent be a problem.

byte[] Serialize()
{
byte[] b = new byte[calculated size];
SomeAppendFuncForByte( Convert.ToByte(memberA) );
SomeAppendFuncForByte( Convert.ToByte(memberB) );
SomeAppendFuncForByte( Convert.ToByte(memberC) );
SomeAppendFuncForByte( Convert.ToByte(memberD) );
SomeAppendFuncForByte( Convert.ToByte(memberE) );
SomeAppendFuncForByte( Convert.ToByte("Test String") );
}

br.Write(Convert.ToByte(serializedArray));

The sizes of each datatype are as follows (for my machine.. should be the same on all x86-Windows boxes). These sizes indicate the number of bytes used to represent them.

unsigned char:1
char:1
unsigned short int:2
unsigned int:4
int:4
unsigned long int:4
long int:4
float:4
double:8
long double:8

g/l :)
Undergamer
Undergamer
The C++ part is easy :)

Write your structure as it would be in C++, then find out how large it will be and hardcode it or use a sizeof(myStruct) statement when reading it like this.

struct myStruct
{
...
}

myStruct myStructInstance;
myFilePointer = fopen("file.dat", "r");
fread(&myStructInstance, sizeof(myStruct), 1, myFilePointer);


----

That struct could just as easily contain this too :)
struct myMonster
{
...
}

struct myStruct
{
long numberOfMonsters;
myMonster *pMonsters;
}

You could access them like this:

for(int i=0; i{
qFunc(myStructInstance.pMonsters);
}

If you make that change however, remember to change your loading code aswell.

int len;
fread(&len, sizeof(int), 1,fp); //Since the first member of myStruct is an integer, grab it first
fseek(fp,0,0);
fread(&myStructInstance, sizeof(myStructInstance) + (len * sizeof(myMonster)), 1,fp);
ArchG
ArchG
Thank you Undergamer for your reply, I tried to make a Function like you said, but I couldn't find a SomeAppendFuncForByte function, so I did it a little differently, and I feel really foolish for doing it such a way..I just want someone to make sure that this will acutally work..the way i'm writing it..

public byte[] Serialize(int id)        {            //Declare all my Byte Arrays            byte[] HP, ID, Friendly, Sight, ObjectDropped1, ObjectDropped2, RespawnRate, Speed,                    Strength, Value1, Value2;            byte[] Total = new Byte[41];            //Convert Everything To Their Own Byte Arrays            HP = System.BitConverter.GetBytes(Monster[id].HP);            ID = BitConverter.GetBytes(Monster[id].ID);            Friendly = BitConverter.GetBytes(Monster[id].Friendly);            Sight = BitConverter.GetBytes(Monster[id].Sight);            ObjectDropped1 = BitConverter.GetBytes(Monster[id].ObjectDropped1);            ObjectDropped2 = BitConverter.GetBytes(Monster[id].ObjectDropped2);            RespawnRate = BitConverter.GetBytes(Monster[id].RespawnRate);            Speed = BitConverter.GetBytes(Monster[id].Speed);            Strength = BitConverter.GetBytes(Monster[id].Strength);            Value1 = BitConverter.GetBytes(Monster[id].Value1);            Value2 = BitConverter.GetBytes(Monster[id].Value2);                        //Counter to keep track of Total Index            int iCount = 0;            //Add all the Bytes to One Array            for (int i = 0; i <= 3; i++)            {                Total[iCount] = ID;                iCount++;            }            for (int i = 0; i <= 3; i++)            {                Total[iCount] = HP;                iCount++;            }            for (int i = 0; i <= 3; i++)            {                Total[iCount] = Sight;                iCount++;            }            for (int i = 0; i <= 3; i++)            {                Total[iCount] = ObjectDropped1;                iCount++;            }            for (int i = 0; i <= 3; i++)            {                Total[iCount] = ObjectDropped2;                iCount++;            }            for (int i = 0; i <= 3; i++)            {                Total[iCount] = RespawnRate;                iCount++;            }            for (int i = 0; i <= 3; i++)            {                Total[iCount] = Speed;                iCount++;            }            for (int i = 0; i <= 3; i++)            {                Total[iCount] = Strength;                iCount++;            }            for (int i = 0; i <= 3; i++)            {                Total[iCount] = Value1;                iCount++;            }            for (int i = 0; i <= 3; i++)            {                Total[iCount] = Value2;                iCount++;            }            Total[iCount] = Friendly[0];            return Total;        }

I know there MUST be a better way to writing that than I did..but I guess if it works it works..right?..So anyway, if anyone can see any problems with that, please let me know
(also a cleaner way of writing it would be useful too ;-))
Thanks Again

ArchG
mutex
mutex
Wouldn't it be easier to just use BinaryWriter to write the individual elements of the struct?

struct MonsterStruct{  // fields...  void Serialize(BinaryWriter w)  {    w.Write(sName);    w.Write(ID);    // etc.  }}FileStream fs = new FileStream("Monsters.dat", FileMode.CreateNew);BinaryWriter w = new BinaryWriter(fs);MonsterStruct s;s.Serialize(w);
You can't serialize the struct with one call, I don't think, hence the need to serialize its fields individually. Additionally, the documentation is unclear about the size of the length prefix for strings, stating that it's either a byte or a word.

Undergamer
Undergamer
Quote:
Original post by ArchG
Thank you Undergamer for your reply, I tried to make a Function like you said, but I couldn't find a SomeAppendFuncForByte function, so I did it a little ...


Sorry about that ;( I was getting ready for work at the time, otherwise I would have copied a relevant function. All I meant by that was a function that did this :)

I know the code is messy, but its C++ and I've been working mostly with C# and it does 99% of that stuff for you :)

void AppendByte(BYTE* pByteArray, BYTE* newByte)
{
BYTE *newByteArray = new BYTE[ sizeof(pByteArray) + 1 ];
newByteArray[sizeof(pByteArray)] = newByte;
memcpy(newByteArray, pByteArray);
delete pByteArray;
pByteArray = newByteArray;
}
Undergamer
Undergamer
Quote:
Original post by mutex
Wouldn't it be easier to just use BinaryWriter to write the individual elements of the struct?

*** Source Snippet Removed ***You can't serialize the struct with one call, I don't think, hence the need to serialize its fields individually. Additionally, the documentation is unclear about the size of the length prefix for strings, stating that it's either a byte or a word.


Thats fine and well, and I think your approach is simpler for this; however, what I was getting at was generating the data and keeping it in memory. That way if you needed to transfer that info over say a network or you needed to encrypt or hash the information you could do it directly from that byte array without the need of having to read it back from the file:

Essentially its this:
Class -> Serialize -> Byte array of Serialized Data -> Hash/Crypto/Network -> File

Instead of:
Class -> Serialize -> File -> Read Byte Array -> Hash/Crypto/Network -> File


---

In .NET you could do this:

using System.Cryptographyvoid SomeFunc(){ //... Assuming you serialized everything already into a byte array ... SHA1CryptoProvider sha = new SHA1CryptoProvider(); byte[] myHash = sha.ComputeHash(mySerializedByteArray);}


This would give you the hash of the file before it was written, possibly making realtime checking of the hash a bit easier. You wouldent want your clients changing monster data would you :). The SHA hash could also be stored first in the file or somewhere on a server. You could pick and choose which part of the file to hash aswell.
VizOne
VizOne
Have a look at the System.Runtime.InteropServices.Marshal class. It provides a lot of usefull functions for interoperability.

Marshalling blittable types (int, float etc) is easy. However, other types - like strings - require a bit more work.

For example, in your case you could marshal the struct like this:

// force "nice" memory layout, marshal strings as ANSI strings[StructLayout(LayoutKind.Sequential, Pack=4, CharSet = CharSet.Ansi)]struct MonsterStruct {	// marshal as a constant size array (char[128])	[MarshalAs(UnmanagedType.ByValTStr, SizeConst=128)]	public string sName;	public int ID;	public int HP;	public int Sight;	public int Speed;	// ...	// writes this instance to disc	public void Save(string fileName) {				// first, get the unmanaged size of the structur					int size = Marshal.SizeOf(typeof(MonsterStruct));				// allocate an unmanaged buffer of the right size		IntPtr data = Marshal.AllocCoTaskMem(size);		try {					// marshal structure to unmanaged memory			// this step will copy all blittable members			// For the string-member, this will re-encode			// the string to a ANSI string, truncate it to 127			// characters if necessary (buffer is 128 = 127 + zero byte)			Marshal.StructureToPtr(this, data, false);						// create managed buffer and copy marshalled data there			byte[] buffer = new byte[size];			Marshal.Copy(data, buffer, 0, buffer.Length);						// write buffer using the simple stream functions			using(Stream s = File.Open(fileName, FileMode.Truncate, FileAccess.Write)) {				s.Write(buffer, 0, buffer.Length);								}		} finally {			Marshal.FreeCoTaskMem(data);					}	}				public static MonsterStruct Load(string fileName) {		// first, get the unmanaged size of the structur					int size = Marshal.SizeOf(typeof(MonsterStruct));				// allocate a buffer that can hold the native image of the struct		byte[] buffer = new byte[size];				// read data from file		using(Stream s = File.OpenRead(fileName)) {			s.Read(buffer, 0, size);		}				// allocate unmanaged memory		IntPtr ptr = Marshal.AllocCoTaskMem(buffer.Length);		try {			// copy buffer to unmanaged memory			Marshal.Copy(buffer, 0, ptr,  buffer.Length);			// marshal to MonsterStruct:			object ms = Marshal.PtrToStructure(ptr, typeof(MonsterStruct));			return (MonsterStruct)ms;		} finally {			Marshal.FreeCoTaskMem(ptr);		}					}};class Program	{					static void Main() {			// create a struct		MonsterStruct ms = new MonsterStruct();		ms.sName = "This is a very simple test!";		ms.ID= 1234;		ms.HP = 12;		ms.Sight = 20;		ms.Speed = 5;				// save it		ms.Save("c:\\struct.bin");				// load the same struct from disc again:		MonsterStruct loadedMS = MonsterStruct.Load("c:\\struct.bin");		// show on console:		Console.WriteLine(loadedMS.sName);		Console.WriteLine(loadedMS.ID);		Console.WriteLine(loadedMS.HP);		Console.WriteLine(loadedMS.Sight);		Console.WriteLine(loadedMS.Speed);					Console.ReadLine();				}}


To make the code more efficient, you can use unsafe/fixed in this context:

public void Save(string fileName) {				// first, get the unmanaged size of the structur					int size = Marshal.SizeOf(typeof(MonsterStruct));				// allocate a memory buffer		byte[] buffer = new byte[size];				unsafe {			// pin the data array and get a pointer to the first element			fixed(byte * ptrBuffer = &buffer[0]) {				// marshal structure to memory				Marshal.StructureToPtr(this, new IntPtr(ptrBuffer), false);			}		}							// write buffer using the simple stream functions		using(Stream s = File.Open(fileName, FileMode.Truncate, FileAccess.Write)) {			s.Write(buffer, 0, buffer.Length);							}	}				public static MonsterStruct Load(string fileName) {			// first, get the unmanaged size of the structur					int size = Marshal.SizeOf(typeof(MonsterStruct));				// allocate a buffer that can hold the whole file					byte[] buffer = new byte[size];				// read data from file		using(Stream s = File.OpenRead(fileName)) {			s.Read(buffer, 0, size);		}				unsafe {			// pin the data array and get a pointer to the first element			fixed(byte * ptrBuffer = &buffer[0]) {				// marshal the structure to the byte array				object ms = Marshal.PtrToStructure(new IntPtr(ptrBuffer), typeof(MonsterStruct));				// all done :)				return (MonsterStruct)ms;			}		}		}


Hope that helps.

Oh, btw, on the native side, you can use this layout of the structure:

struct MonsterStruct {  char sName[128];  int ID;  int HP;  int Sight;  int Speed;};


Be sure to set the member alignment to 4 bytes though!


Regards,
Andre
Washu
Washu
Quote:
Original post by mutex
Wouldn't it be easier to just use BinaryWriter to write the individual elements of the struct?

*** Source Snippet Removed ***You can't serialize the struct with one call, I don't think, hence the need to serialize its fields individually. Additionally, the documentation is unclear about the size of the length prefix for strings, stating that it's either a byte or a word.


A string is prefixed by a 7 bit encoded integer representing the length of the string. The reason they say it can be a byte or a word is because those are the most likely sizes. However, if you do the math, you will quickly see that you can generate numbers that are 5 bytes in length (would be an awfuly LONG string mind you, over 2^31 characters in length.)
In time the project grows, the ignorance of its devs it shows, with many a convoluted function, it plunges into deep compunction, the price of failure is high, Washu's mirth is nigh.
mutex
mutex
Quote:
Original post by Undergamer
Thats fine and well, and I think your approach is simpler for this; however, what I was getting at was generating the data and keeping it in memory. That way if you needed to transfer that info over say a network or you needed to encrypt or hash the information you could do it directly from that byte array without the need of having to read it back from the file:

Essentially its this:
Class -> Serialize -> Byte array of Serialized Data -> Hash/Crypto/Network -> File

Instead of:
Class -> Serialize -> File -> Read Byte Array -> Hash/Crypto/Network -> File
How about using MemoryStream and writing to it using BinaryWriter? Then you can access the byte array via MemoryStream.GetBuffer. Of course, this might be more inefficient than other methods, but for plain simplicity it looks good to me.

Gambit007
Gambit007
BitConverter.GetBytes works to get the bytes from an object
Undergamer
Undergamer
Quote:
Original post by Gambit007
BitConverter.GetBytes works to get the bytes from an object


I like it :)
reaptide
reaptide
Here's an article that I found very useful when I started writing the virtual file system for my program.

C# and advanced binary files

The RawSerialize() and RawDeserialize() methods were a big help. As an added bonus you don't even need to use a BinaryWriter. You can use the resulting array of bytes directly in FileStream's Read and Write methods.
ArchG
ArchG
Thank You everyone for all your help. I think I did almost every method on here, I think the easiest one though was using those premade functions provided in the article that reaptide posted...Incase anyone else has this same question ...i'll Post how I did it.


Part 1 - C# - Writing the Binary File



This right here is just my structure that I will soon be saving to a binary file
[StructLayout(LayoutKind.Sequential)] //This line is needed as this tells how the data should be stored (Sequential)    public struct sMonster    {        //This next line is here because strings have an undefined size..so here i specify that I want the size to be 32 bytes        [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 32)]         public string sName;        public Int32 ID;        public Int32 HP;        public Int32 Sight;        public Int32 Strength;        public Int32 ObjectDropped1;        public Int32 ObjectDropped2;        public Int32 Value1;        public Int32 Value2;        public bool Friendly;        public Int32 Speed;        public Int32 RespawnRate;    };


Here are the two handy functions I used (taken from the article..they work very similar to the ones that VizOne posted as well (take a look at his post)
//This Function takes any structure, and will return an array of bytes, that can be nicely written to a file via the BinaryWriter     public static byte[] RawSerialize(object anything)        {            int rawsize = Marshal.SizeOf(anything);            IntPtr buffer = Marshal.AllocHGlobal(rawsize);            Marshal.StructureToPtr(anything, buffer, false);            byte[] rawdatas = new byte[rawsize];            Marshal.Copy(buffer, rawdatas, 0, rawsize);            Marshal.FreeHGlobal(buffer);            return rawdatas;        }//This function is the opposite, it will take an array of bytes, and turn it back into the structure (a typecast is needed although)        public static object RawDeserialize(byte[] rawdatas, Type anytype)        {            int rawsize = Marshal.SizeOf(anytype);            if (rawsize > rawdatas.Length)                return null;            IntPtr buffer = Marshal.AllocHGlobal(rawsize);            Marshal.Copy(rawdatas, 0, buffer, rawsize);            object retobj = Marshal.PtrToStructure(buffer, anytype);            Marshal.FreeHGlobal(buffer);            return retobj;        }


Now that I have the structure (Monster Structure) and the functions to turn it into an array of bytes, it can be easily written to a binary file
//Monster = a pre initialized Monster structureconst string FILE_NAME = "Monsters.dat";FileStream fs = new FileStream(FILE_NAME, FileMode.Create);BinaryWriter w = new BinaryWriter(fs);byte[] buffer = RawSerialize(Monster);w.Write(buffer);//cleanupw.Close();fs.Close();

And that's all there is to it!


Part 2 - C++ - Reading the Binary File



This Part like Undergamer said, is very easy, there's nothing special needed, just like reading any file
//My structure..-notice i use a character array the same size as my string in the C# structure	struct sMonster{			 char cName[32];			 int ID;			 int HP;			 int Sight;                         int Strength;			 int ObjectDropped1;                         int ObjectDropped2;			 int Value1;                         int Value2;			 bool Friendly;			 int Speed;			 int RespawnRate;          	};sMonster Monster;//FilePointerFILE * fp;fp = fopen(FILE_NAME, "r");fread(&Monster, sizeof(sMonster), 1, fp);fclose(fp);


And that's all there is to it, thats to everyone who helped. Ratings to all ;-)

ArchG
Ferneu
Ferneu
Since the page posted by reaptide do not seem to be working right now, I'd like to say thanks to you all, specially for ArchG for reposting the code here. It has helped me a lot!
pdvbalaji
pdvbalaji

Hi,
Thank u giveing this coding ,I use this code in C#.net using windows application.
But I am getting error
Error: type project Name cannot be marshaled as an unmnaged structure;no meaning full size of offset can be computed.

Please solved this problem.
I am waiting for u r replay
I am mail id is pdvbalaji@gmail.com
Codeka
Codeka
Quote:
Original post by pdvbalaji
Thank u giveing this coding ,I use this code in C#.net using windows application.
But I am getting error
Error: type project Name cannot be marshaled as an unmnaged structure;no meaning full size of offset can be computed.
What does your code look like, and where does the error happen?
slewrate74
slewrate74
I am trying to do something similar as TS. However, I want to include bitmap data into my structure. Is it possible to accomplish that even if the bitmaps vary in size ???
ArchG
ArchG
wow, was very surprised to just browse the .net forum and see a thread i created 4 and a half years ago pop up :-).

I think you could go about it in a couple ways. I would probably write the size of the bitmap (number of bytes) as an integer, so you could read that, and know how many bytes the bitmap as, and just write the bitmap as some sort of serialized byte array. I think .net has some easier methods for serializing data, I'd have to check, but I know that it is possible.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.