[.net] Writing Binary Files in C#, Reading them in C++

Started by
17 comments, last by ArchG 14 years, 4 months ago
Hello, i'm working on a small game in C++ right now, and I needed a MonsterObject Creator, so since C# seems faster and easier for such a thing, I decided to make the MonsterObject Creator with C#.. Right now I have a monster structure, like this..

struct MonsterStruct
{
string sName;
int ID;
int HP;
int Sight;
int Speed;
bool bFriendly;
float RespawnRate;
int ObjectDropped1;
float fValue1;
int ObjectDropped2;
float fValue2;
};
MonsterStruct[] Monster;
Monster = new MonsterStruct[255];

Im wondering what the best way is to write a bunch of structures like that to a binary file that can be read in C++..I figured a BinaryWriter was really the only way to go..

//Pretend Monster is a MonsterStruct array of 255 and is already created and filled
FileStream fs = new FileStream("Monsters.dat", FileMode.CreateNew);
BinaryWriter w = new BinaryWriter(fs);
//What would be the easest way to write it?
w.Write(??);

And Then a bonus...how would I read that in C++ (I guess it's hard to know how to read it until you know how your writing it though huh?)
Advertisement
I would create a method in your C# class which is called "byte[] Serialize()". Using the array classes already in C# you can append the bytes for each field of the class so that they match up with what would be read in C++. There are different datatypes and assuming that you store it in order the correct way it shouldent be a problem.

byte[] Serialize()
{
byte[] b = new byte[calculated size];
SomeAppendFuncForByte( Convert.ToByte(memberA) );
SomeAppendFuncForByte( Convert.ToByte(memberB) );
SomeAppendFuncForByte( Convert.ToByte(memberC) );
SomeAppendFuncForByte( Convert.ToByte(memberD) );
SomeAppendFuncForByte( Convert.ToByte(memberE) );
SomeAppendFuncForByte( Convert.ToByte("Test String") );
}

br.Write(Convert.ToByte(serializedArray));

The sizes of each datatype are as follows (for my machine.. should be the same on all x86-Windows boxes). These sizes indicate the number of bytes used to represent them.

unsigned char:1
char:1
unsigned short int:2
unsigned int:4
int:4
unsigned long int:4
long int:4
float:4
double:8
long double:8

g/l :)
The C++ part is easy :)

Write your structure as it would be in C++, then find out how large it will be and hardcode it or use a sizeof(myStruct) statement when reading it like this.

struct myStruct
{
...
}

myStruct myStructInstance;
myFilePointer = fopen("file.dat", "r");
fread(&myStructInstance, sizeof(myStruct), 1, myFilePointer);


----

That struct could just as easily contain this too :)
struct myMonster
{
...
}

struct myStruct
{
long numberOfMonsters;
myMonster *pMonsters;
}

You could access them like this:

for(int i=0; i<myStructInstance.numberOfMonsters; i++)
{
qFunc(myStructInstance.pMonsters);
}

If you make that change however, remember to change your loading code aswell.

int len;
fread(&len, sizeof(int), 1,fp); //Since the first member of myStruct is an integer, grab it first
fseek(fp,0,0);
fread(&myStructInstance, sizeof(myStructInstance) + (len * sizeof(myMonster)), 1,fp);
Thank you Undergamer for your reply, I tried to make a Function like you said, but I couldn't find a SomeAppendFuncForByte function, so I did it a little differently, and I feel really foolish for doing it such a way..I just want someone to make sure that this will acutally work..the way i'm writing it..

public byte[] Serialize(int id)        {            //Declare all my Byte Arrays            byte[] HP, ID, Friendly, Sight, ObjectDropped1, ObjectDropped2, RespawnRate, Speed,                    Strength, Value1, Value2;            byte[] Total = new Byte[41];            //Convert Everything To Their Own Byte Arrays            HP = System.BitConverter.GetBytes(Monster[id].HP);            ID = BitConverter.GetBytes(Monster[id].ID);            Friendly = BitConverter.GetBytes(Monster[id].Friendly);            Sight = BitConverter.GetBytes(Monster[id].Sight);            ObjectDropped1 = BitConverter.GetBytes(Monster[id].ObjectDropped1);            ObjectDropped2 = BitConverter.GetBytes(Monster[id].ObjectDropped2);            RespawnRate = BitConverter.GetBytes(Monster[id].RespawnRate);            Speed = BitConverter.GetBytes(Monster[id].Speed);            Strength = BitConverter.GetBytes(Monster[id].Strength);            Value1 = BitConverter.GetBytes(Monster[id].Value1);            Value2 = BitConverter.GetBytes(Monster[id].Value2);                        //Counter to keep track of Total Index            int iCount = 0;            //Add all the Bytes to One Array            for (int i = 0; i <= 3; i++)            {                Total[iCount] = ID;                iCount++;            }            for (int i = 0; i <= 3; i++)            {                Total[iCount] = HP;                iCount++;            }            for (int i = 0; i <= 3; i++)            {                Total[iCount] = Sight;                iCount++;            }            for (int i = 0; i <= 3; i++)            {                Total[iCount] = ObjectDropped1;                iCount++;            }            for (int i = 0; i <= 3; i++)            {                Total[iCount] = ObjectDropped2;                iCount++;            }            for (int i = 0; i <= 3; i++)            {                Total[iCount] = RespawnRate;                iCount++;            }            for (int i = 0; i <= 3; i++)            {                Total[iCount] = Speed;                iCount++;            }            for (int i = 0; i <= 3; i++)            {                Total[iCount] = Strength;                iCount++;            }            for (int i = 0; i <= 3; i++)            {                Total[iCount] = Value1;                iCount++;            }            for (int i = 0; i <= 3; i++)            {                Total[iCount] = Value2;                iCount++;            }            Total[iCount] = Friendly[0];            return Total;        }

I know there MUST be a better way to writing that than I did..but I guess if it works it works..right?..So anyway, if anyone can see any problems with that, please let me know
(also a cleaner way of writing it would be useful too ;-))
Thanks Again

ArchG
Wouldn't it be easier to just use BinaryWriter to write the individual elements of the struct?

struct MonsterStruct{  // fields...  void Serialize(BinaryWriter w)  {    w.Write(sName);    w.Write(ID);    // etc.  }}FileStream fs = new FileStream("Monsters.dat", FileMode.CreateNew);BinaryWriter w = new BinaryWriter(fs);MonsterStruct s;s.Serialize(w);
You can't serialize the struct with one call, I don't think, hence the need to serialize its fields individually. Additionally, the documentation is unclear about the size of the length prefix for strings, stating that it's either a byte or a word.

Quote:Original post by ArchG
Thank you Undergamer for your reply, I tried to make a Function like you said, but I couldn't find a SomeAppendFuncForByte function, so I did it a little ...


Sorry about that ;( I was getting ready for work at the time, otherwise I would have copied a relevant function. All I meant by that was a function that did this :)

I know the code is messy, but its C++ and I've been working mostly with C# and it does 99% of that stuff for you :)

void AppendByte(BYTE* pByteArray, BYTE* newByte)
{
BYTE *newByteArray = new BYTE[ sizeof(pByteArray) + 1 ];
newByteArray[sizeof(pByteArray)] = newByte;
memcpy(newByteArray, pByteArray);
delete pByteArray;
pByteArray = newByteArray;
}
Quote:Original post by mutex
Wouldn't it be easier to just use BinaryWriter to write the individual elements of the struct?

*** Source Snippet Removed ***You can't serialize the struct with one call, I don't think, hence the need to serialize its fields individually. Additionally, the documentation is unclear about the size of the length prefix for strings, stating that it's either a byte or a word.


Thats fine and well, and I think your approach is simpler for this; however, what I was getting at was generating the data and keeping it in memory. That way if you needed to transfer that info over say a network or you needed to encrypt or hash the information you could do it directly from that byte array without the need of having to read it back from the file:

Essentially its this:
Class -> Serialize -> Byte array of Serialized Data -> Hash/Crypto/Network -> File

Instead of:
Class -> Serialize -> File -> Read Byte Array -> Hash/Crypto/Network -> File


---

In .NET you could do this:

using System.Cryptographyvoid SomeFunc(){ //... Assuming you serialized everything already into a byte array ... SHA1CryptoProvider sha = new SHA1CryptoProvider(); byte[] myHash = sha.ComputeHash(mySerializedByteArray);}


This would give you the hash of the file before it was written, possibly making realtime checking of the hash a bit easier. You wouldent want your clients changing monster data would you :). The SHA hash could also be stored first in the file or somewhere on a server. You could pick and choose which part of the file to hash aswell.
Have a look at the System.Runtime.InteropServices.Marshal class. It provides a lot of usefull functions for interoperability.

Marshalling blittable types (int, float etc) is easy. However, other types - like strings - require a bit more work.

For example, in your case you could marshal the struct like this:

// force "nice" memory layout, marshal strings as ANSI strings[StructLayout(LayoutKind.Sequential, Pack=4, CharSet = CharSet.Ansi)]struct MonsterStruct {	// marshal as a constant size array (char[128])	[MarshalAs(UnmanagedType.ByValTStr, SizeConst=128)]	public string sName;	public int ID;	public int HP;	public int Sight;	public int Speed;	// ...	// writes this instance to disc	public void Save(string fileName) {				// first, get the unmanaged size of the structur					int size = Marshal.SizeOf(typeof(MonsterStruct));				// allocate an unmanaged buffer of the right size		IntPtr data = Marshal.AllocCoTaskMem(size);		try {					// marshal structure to unmanaged memory			// this step will copy all blittable members			// For the string-member, this will re-encode			// the string to a ANSI string, truncate it to 127			// characters if necessary (buffer is 128 = 127 + zero byte)			Marshal.StructureToPtr(this, data, false);						// create managed buffer and copy marshalled data there			byte[] buffer = new byte[size];			Marshal.Copy(data, buffer, 0, buffer.Length);						// write buffer using the simple stream functions			using(Stream s = File.Open(fileName, FileMode.Truncate, FileAccess.Write)) {				s.Write(buffer, 0, buffer.Length);								}		} finally {			Marshal.FreeCoTaskMem(data);					}	}				public static MonsterStruct Load(string fileName) {		// first, get the unmanaged size of the structur					int size = Marshal.SizeOf(typeof(MonsterStruct));				// allocate a buffer that can hold the native image of the struct		byte[] buffer = new byte[size];				// read data from file		using(Stream s = File.OpenRead(fileName)) {			s.Read(buffer, 0, size);		}				// allocate unmanaged memory		IntPtr ptr = Marshal.AllocCoTaskMem(buffer.Length);		try {			// copy buffer to unmanaged memory			Marshal.Copy(buffer, 0, ptr,  buffer.Length);			// marshal to MonsterStruct:			object ms = Marshal.PtrToStructure(ptr, typeof(MonsterStruct));			return (MonsterStruct)ms;		} finally {			Marshal.FreeCoTaskMem(ptr);		}					}};class Program	{					static void Main() {			// create a struct		MonsterStruct ms = new MonsterStruct();		ms.sName = "This is a very simple test!";		ms.ID= 1234;		ms.HP = 12;		ms.Sight = 20;		ms.Speed = 5;				// save it		ms.Save("c:\\struct.bin");				// load the same struct from disc again:		MonsterStruct loadedMS = MonsterStruct.Load("c:\\struct.bin");		// show on console:		Console.WriteLine(loadedMS.sName);		Console.WriteLine(loadedMS.ID);		Console.WriteLine(loadedMS.HP);		Console.WriteLine(loadedMS.Sight);		Console.WriteLine(loadedMS.Speed);					Console.ReadLine();				}}


To make the code more efficient, you can use unsafe/fixed in this context:

public void Save(string fileName) {				// first, get the unmanaged size of the structur					int size = Marshal.SizeOf(typeof(MonsterStruct));				// allocate a memory buffer		byte[] buffer = new byte[size];				unsafe {			// pin the data array and get a pointer to the first element			fixed(byte * ptrBuffer = &buffer[0]) {				// marshal structure to memory				Marshal.StructureToPtr(this, new IntPtr(ptrBuffer), false);			}		}							// write buffer using the simple stream functions		using(Stream s = File.Open(fileName, FileMode.Truncate, FileAccess.Write)) {			s.Write(buffer, 0, buffer.Length);							}	}				public static MonsterStruct Load(string fileName) {			// first, get the unmanaged size of the structur					int size = Marshal.SizeOf(typeof(MonsterStruct));				// allocate a buffer that can hold the whole file					byte[] buffer = new byte[size];				// read data from file		using(Stream s = File.OpenRead(fileName)) {			s.Read(buffer, 0, size);		}				unsafe {			// pin the data array and get a pointer to the first element			fixed(byte * ptrBuffer = &buffer[0]) {				// marshal the structure to the byte array				object ms = Marshal.PtrToStructure(new IntPtr(ptrBuffer), typeof(MonsterStruct));				// all done :)				return (MonsterStruct)ms;			}		}		}


Hope that helps.

Oh, btw, on the native side, you can use this layout of the structure:

struct MonsterStruct {  char sName[128];  int ID;  int HP;  int Sight;  int Speed;};


Be sure to set the member alignment to 4 bytes though!


Regards,
Andre
Andre Loker | Personal blog on .NET
Quote:Original post by mutex
Wouldn't it be easier to just use BinaryWriter to write the individual elements of the struct?

*** Source Snippet Removed ***You can't serialize the struct with one call, I don't think, hence the need to serialize its fields individually. Additionally, the documentation is unclear about the size of the length prefix for strings, stating that it's either a byte or a word.


A string is prefixed by a 7 bit encoded integer representing the length of the string. The reason they say it can be a byte or a word is because those are the most likely sizes. However, if you do the math, you will quickly see that you can generate numbers that are 5 bytes in length (would be an awfuly LONG string mind you, over 2^31 characters in length.)

In time the project grows, the ignorance of its devs it shows, with many a convoluted function, it plunges into deep compunction, the price of failure is high, Washu's mirth is nigh.

Quote:Original post by Undergamer
Thats fine and well, and I think your approach is simpler for this; however, what I was getting at was generating the data and keeping it in memory. That way if you needed to transfer that info over say a network or you needed to encrypt or hash the information you could do it directly from that byte array without the need of having to read it back from the file:

Essentially its this:
Class -> Serialize -> Byte array of Serialized Data -> Hash/Crypto/Network -> File

Instead of:
Class -> Serialize -> File -> Read Byte Array -> Hash/Crypto/Network -> File
How about using MemoryStream and writing to it using BinaryWriter? Then you can access the byte array via MemoryStream.GetBuffer. Of course, this might be more inefficient than other methods, but for plain simplicity it looks good to me.

This topic is closed to new replies.

Advertisement