Jump to content
  • Advertisement
cozzie

C++ Writing char arrays (binary)

Recommended Posts

Hi all,

For practice I'm trying to write 3D mesh data to a binary format. To eventually also be able to read meshes from binary files (provided by the asset pipeline).

I understand how to write floats, vectors of floats, ints etc by using a write((char*)myVar, sizeof(type) to the file, for which I'm using std::ofstream.

My goal is to write a 3D vector or other type (struct) to the file and be able to read it back the same way. From what I understand, I need to create function that returns a char array combining all members of the struct.

So in short; how can I create this char array, in a "safe" way?

So far I've come up with this approach:

- calculate number of needed bytes/chars, i.e 12 bytes for a 3D vector with 3 floats

- char result[12]

- cast the 3 individual floats to char[4]'s and use strcpy to result for the 1st and strcat for the others

Is this the way to go or would you suggest other approaches?

question 2; How would I read/convert the char array back to the 3 floats when reading it from the binary file?

Any input is appreciated.

Share this post


Link to post
Share on other sites
Advertisement

I'm not even sure what's your problem there. Let's start from the basics: the only thing that truly exist is memory and bits. As long as you're going with 'basic' types such as floats the following will do: 

ubyte blobby[12];
memcpy(blobby, src, sizeof(blobby));
float *bruh = (float*)blobby;

Hoping `blobby` to be properly aligned.

First you have to ensure your `int` is the same as the serialized `int` and that's why you have `__int32` or `std::int64_t`. And of course you have endianess problems but let's assume you'll be loading on 'coherent' machines.

When `struct` enters the picture `memcpy` goes out due to possibility of different compiler packing.

There are various options for (de)serialization, I'd suggest pupping all the things.

That or google protobuffers but in my experience they're not quite convenient for game purposes due to their everything can be optional nonsense.

 

 

Share this post


Link to post
Share on other sites

Thanks guys.

I've been playing around using this input. Unfortunately writing the struct or array (std::vector) at once, doesn't work by trying to cast to (myType*), it does work when I cast it to (char*). See the code samples below, tested all 3 positive.

Would I have any risks on this approach, depending on platform? (size of float etc.).
I'll make sure the struct I use as a POD struct, which is the case in the example below.

void ReadWriteStruct()
{
	std::ofstream writeFile;
	writeFile.open("data.bin", std::ios::out, std::ios::binary);
	writeFile.write((char*)&srcVector, sizeof(VECTOR3));
	writeFile.close();

	MessageBox(NULL, L"Vector written to binary file - full struct!", L"Done", MB_OK);

	std::ifstream readFile;
	readFile.open("data.bin", std::ios::in, std::ios::binary);

	VECTOR3 myReadVec3;
	readFile.read((char*)&myReadVec3, sizeof(VECTOR3));
	readFile.close();

	char tempText[100];
	sprintf_s(tempText, "Read: %f, %f, %f\n", myReadVec3.x, myReadVec3.y, myReadVec3.z);
	OutputDebugStringA(tempText);
}

void ReadWriteArray()
{
	std::vector<VECTOR3> myVectors(2);
	myVectors[0].x = 0.2f;
	myVectors[0].y = 0.7f;
	myVectors[0].z = 0.95f;
	myVectors[1].x = 5.2f;
	myVectors[1].y = 4.7f;
	myVectors[1].z = 7.75f;

	std::ofstream writeFile;
	writeFile.open("data.bin", std::ios::out, std::ios::binary);
	writeFile.write((char*)&myVectors[0], myVectors.size() * sizeof(VECTOR3));
	writeFile.close();

	MessageBox(NULL, L"Vector written to binary file - full struct!", L"Done", MB_OK);

	std::ifstream readFile;
	readFile.open("data.bin", std::ios::in, std::ios::binary);

	std::vector<VECTOR3> readVectors(2);
	readFile.read((char*)&readVectors[0], sizeof(VECTOR3));
	readFile.read((char*)&readVectors[1], sizeof(VECTOR3));
	readFile.close();

	char tempText[100];
	sprintf_s(tempText, "Read 1: %f, %f, %f\n", readVectors[0].x, readVectors[0].y, readVectors[0].z);
	OutputDebugStringA(tempText);
	sprintf_s(tempText, "Read 2: %f, %f, %f\n", readVectors[1].x, readVectors[1].y, readVectors[1].z);
	OutputDebugStringA(tempText);
}

 

Share this post


Link to post
Share on other sites
17 hours ago, cozzie said:

Thanks guys.

I've been playing around using this input. Unfortunately writing the struct or array (std::vector) at once, doesn't work by trying to cast to (myType*), it does work when I cast it to (char*). See the code samples below, tested all 3 positive.

Would I have any risks on this approach, depending on platform? (size of float etc.).
I'll make sure the struct I use as a POD struct, which is the case in the example below.


void ReadWriteStruct()
{
	std::ofstream writeFile;
	writeFile.open("data.bin", std::ios::out, std::ios::binary);
	writeFile.write((char*)&srcVector, sizeof(VECTOR3));
	writeFile.close();

	MessageBox(NULL, L"Vector written to binary file - full struct!", L"Done", MB_OK);

	std::ifstream readFile;
	readFile.open("data.bin", std::ios::in, std::ios::binary);

	VECTOR3 myReadVec3;
	readFile.read((char*)&myReadVec3, sizeof(VECTOR3));
	readFile.close();

	char tempText[100];
	sprintf_s(tempText, "Read: %f, %f, %f\n", myReadVec3.x, myReadVec3.y, myReadVec3.z);
	OutputDebugStringA(tempText);
}

void ReadWriteArray()
{
	std::vector<VECTOR3> myVectors(2);
	myVectors[0].x = 0.2f;
	myVectors[0].y = 0.7f;
	myVectors[0].z = 0.95f;
	myVectors[1].x = 5.2f;
	myVectors[1].y = 4.7f;
	myVectors[1].z = 7.75f;

	std::ofstream writeFile;
	writeFile.open("data.bin", std::ios::out, std::ios::binary);
	writeFile.write((char*)&myVectors[0], myVectors.size() * sizeof(VECTOR3));
	writeFile.close();

	MessageBox(NULL, L"Vector written to binary file - full struct!", L"Done", MB_OK);

	std::ifstream readFile;
	readFile.open("data.bin", std::ios::in, std::ios::binary);

	std::vector<VECTOR3> readVectors(2);
	readFile.read((char*)&readVectors[0], sizeof(VECTOR3));
	readFile.read((char*)&readVectors[1], sizeof(VECTOR3));
	readFile.close();

	char tempText[100];
	sprintf_s(tempText, "Read 1: %f, %f, %f\n", readVectors[0].x, readVectors[0].y, readVectors[0].z);
	OutputDebugStringA(tempText);
	sprintf_s(tempText, "Read 2: %f, %f, %f\n", readVectors[1].x, readVectors[1].y, readVectors[1].z);
	OutputDebugStringA(tempText);
}

 

I made a mistake, it should have been char* not myType*. I don't see a structure in your post but usually a vector struct is a POD.

You can read the vector array in a single call as well.

Krohm already posted the problems you may face.

Share this post


Link to post
Share on other sites

Thanks. The VECTOR3 is my struct, so it's working for both an individual struct as a std::vector of them. Thanks

Edited by cozzie

Share this post


Link to post
Share on other sites
On 10/11/2017 at 12:57 AM, Krohm said:

There are various options for (de)serialization, I'd suggest pupping all the things.

I second this approach.

Break it down so that you're only serializing fundamental types. This allows you to avoid all issues with alignment and padding, properly handle endian-ness, and trivially support composites of serializeable types.

There's nothing you gain from trying to read/write entire buffers of typed data at once, other than the potential for hard-to-find bugs.

Share this post


Link to post
Share on other sites

A little bit off topic but i always first writr then length of char array and then i writr char array anyway

Memcpy is your pal, you just memcpy(&floatvar, &chartable[index],3);

 

Something kike this you copy some amount memory to float pointer

Share this post


Link to post
Share on other sites

Do you mean first writing everything to a char array (aka buffer) and then write that buffer to file?

That could work, but then I have to calculate the index manually, which could be a risk if some type has a different bytesize on another problem (writing per variable and using sizeof, I think prevents this). In your example, shouldn't 3 be 4 (bytes for a float)?

But perhaps there's a way to use a buffer while keeping this in mind (or don't manually calculate the index of each variable).

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

Participate in the game development conversation and more when you create an account on GameDev.net!

Sign me up!