reading float from binary file (endian problem?)

Started by
13 comments, last by frob 9 years, 7 months ago

So I've been working on writing and reading binary file, and I'm surprised that I have gone for as long as I did without any big problems, but I seem to have run into this roadblock that I can't get around. I'm having trouble reading float values from the files I've written, and I'm not exactly sure what causes the problem.


// say this is the file
//  0.0
// 40.0
//  1.0

int main(ig var, nore nvar)
{
	float a, b, c;
	std::ifstream f("myfile.fff", std::ios::binary);
	
	f.read((char*)&a, sizeof(float));
	f.read((char*)&b, sizeof(float));
	f.read((char*)&c, sizeof(float));

	printf("a: %f\nb: %f\nc: %f\n", a, b, c);
}
	

The results I get are:

a: 0 (good, that's what I wanted)

b: 0.0000000000000000000000000000000000000000237211804 (... that's not 40)

c: 0.000000000000000000000000000000000000373583929 (uuugghh)

I searched on google and found that it might be an endian problem and this is where it get confusing for me. I looked in the debugger, changed the value view to 'bytes hex' and I get:

the value of 40: 00 00 20 42

the value of the result: 20 42 00 00

Now because I'm self-taught in c++ there are numerous gaps in my skill set, but I have been understanding that endian-ness is like reading bytes from left to right, or right to left, but this result seems more "shuffled". I also understand that endian-ness differs across processor architectures, so why would this be happening when the file is being written and read on the same machine?

Do I need to fix this with byte swapping?(shuffling) or is there something else that could be the culprit?

Macbook Pro 2.66Ghz dual core, 4GB ram, 512MB vram, MacOSX 9.1, Windows 8.1
Xcode 5.0.2, C++, lua

Advertisement

Show the code that writes it out. Assuming you're writing this out on the same machine as you are reading it from, and assuming you did not apply any transforms to the bytes... No, this would not be an endian problem.

Hard to say without a bit more detail, such as the code for writing it.

In time the project grows, the ignorance of its devs it shows, with many a convoluted function, it plunges into deep compunction, the price of failure is high, Washu's mirth is nigh.



float x = 0.0f, y = 40.0f, z = 1.0f;

void writeTheFile()
{
	std::ofstream f;
	f.open(wFileExt, std::ios::trunc | std::ios::binary | std::ios::out);

	f.write((const char*)&x, sizeof(float));
	f.write((const char*)&y, sizeof(float));
	f.write((const char*)&z, sizeof(float));

	f.close() // did I forget this in the last post? I know it's in the program code...
}

This is basically it.

Macbook Pro 2.66Ghz dual core, 4GB ram, 512MB vram, MacOSX 9.1, Windows 8.1
Xcode 5.0.2, C++, lua


float x = 0.0f, y = 40.0f, z = 1.0f;

void writeTheFile()
{
	std::ofstream f;
	f.open(wFileExt, std::ios::trunc | std::ios::binary | std::ios::out);

	f.write((const char*)&x, sizeof(float));
	f.write((const char*)&y, sizeof(float));
	f.write((const char*)&z, sizeof(float));

	f.close() // did I forget this in the last post? I know it's in the program code...
}

This is basically it.

Basically != It.

The two code snippets, joined together, will operate as expected, which means you are doing something else wrong.


#include <iostream>
#include <fstream>

void writeTheFile()
{
    float x = 0.0f, y = 40.0f, z = 1.0f;
    std::ofstream f("test.bin", std::ios::trunc | std::ios::binary | std::ios::out);

    f.write((const char*)&x, sizeof(float));
    f.write((const char*)&y, sizeof(float));
    f.write((const char*)&z, sizeof(float));
}

void readTheFile() {
    float a, b, c;
    std::ifstream f("test.bin", std::ios::binary);

    f.read((char*)&a, sizeof(float));
    f.read((char*)&b, sizeof(float));
    f.read((char*)&c, sizeof(float));

    std::cout << a << ", " << b << ", " << c << std::endl;
}

int main()
{
    writeTheFile();
    readTheFile();
}

In time the project grows, the ignorance of its devs it shows, with many a convoluted function, it plunges into deep compunction, the price of failure is high, Washu's mirth is nigh.

What happens if you remove the casts to const char* and char*?

 

What happens if you remove the casts to const char* and char*?

The casts are necessary. Read the signatures of the functions (1 2).

Although I would have used reinterpret_cast just to express it better.

In time the project grows, the ignorance of its devs it shows, with many a convoluted function, it plunges into deep compunction, the price of failure is high, Washu's mirth is nigh.

Ah yes, they are indeed necessary. I was on a tablet before, so couldn't immediately check it out, sorry about that.

I now tried the code here (in Visual Studio 2012) and it does indeed work correctly.

NSDuo: maybe you could try the code as Washu posted it ("two snippets together") and see what that gives?

Also, what compiler do you use? Has the file been written on the same system that it was read on?

 

Since the 20 and 42 are in the same order, it's not a byte endian problem. And I doubt you're on a platform that swaps 16-bit words around, either.

The most likely problem is that your reader and writer code don't match the amount of data they're handling. For example, if something in your read code reads an extra two bytes before the float is read, then the file pointer will be in the middle of the float when you try to read it, instead of at the beginning of the float. Ex:



  ???            0.0           40.0          1.0
(other stuff?) | 00 00 00 00 | 00 00 20 42 | 00 00 80 3F |
                                     ^ file pointer before you read your float, so [20 42 | 00 00] will be read (0x00004220) => 2.372118E-41
If you're freading/fwriting whole structs in your code somewhere, then you may want to read up on struct packing: http://stackoverflow.com/questions/3318410/pragma-pack-effect

Since the 20 and 42 are in the same order, it's not a byte endian problem. [...] The most likely problem is that your reader and writer code don't match the amount of data they're handling. [...]

That right there.

Needless to say posting your actual code is necessary because often the bug you experience is not a direct result of the "basically it" code - if it were that easy you wouldn't have asked for help. Usually it is some small detail (sometimes even a typo) that is present in the original code which you are missing. Then it takes all this extra conversation to finally see the real code with the actual problem. If it seems "there is too much code", consider narrowing the problem area down to a small bit of code which reproduces the problem you are experiencing. This could even lead you to identify and debug the issue prior to posting the narrowed down code.

I suppose I was being a little stupid by not posting the full code ( I don't know if it was my ego or whatever...) but I asked for help, so here it is...


int scene::dumpToFile(std::string file)
	{
		
		std::string wFileExt = file;
		wFileExt.append(".sinncascene");
		//wFileExt.append(".txt");
		// open for writing only
		//f.open(wFileExt);
		std::ofstream f;
		f.open(wFileExt, std::ios::trunc | std::ios::binary | std::ios::out);
		if (!f.good())
		{
			printf("The file '%s' was not good", wFileExt.c_str());
			return -1;
		
		} else if (!f.is_open())
		{
			printf("The file '%s' could not be opened", wFileExt.c_str());
			return -1;
		}
		
		// set header
		// 8 bytes
		ui8* header = (ui8*)"sinnca_";
		f.write((char*)header, 8);
		// 2 bytes
		ui16 secondaryHeader = 18420;
		f.write((const char*)&secondaryHeader, 2);
		
		/*
		// set size of file
		ui64 fSize = sizeof(uint) +
					 sizeof(long) +
					 sizeof(entity) * entityRef.size() +
					 sizeof(grid) * gridRef.size() +
					 sizeof(scene) * sceneRef.size() +
					 sizeof(image) * imageRef.size() +
					 sizeof(texture) * textureRef.size();
		// 8 bytes
		f << fSize;
		 */
		
		// write the number of assets
		// 4 bytes
		ui32 noOfAssets = (ui32)(imageRef.size() +
						  textureRef.size());
		f.write((const char*)&noOfAssets, 4);
		
		// write number of nodes
		// 4 bytes
		ui32 noOfNodes = (ui32)(entityRef.size() +
						 gridRef.size() +
						 sceneRef.size());
		f.write((const char*)&noOfNodes, 4);
		
		
		
		/* ********************************************************************************
		 WRITE IMAGES
		 ******************************************************************************* */
		if (imageRef.size() > 0)
		{
			for (int i = 0; i < imageRef.size(); i++)
			{
				// what kind of asset is this? (2 bytes)
				//f.write((const char*) 1, 2);
				f.put((const char) 1); // 1 = image
				
				// what number asset are we at? (2 bytes)
				f.write((const char*) &i, 2);
				
				// what is the size of this asset? (2 bytes)
				ui16 sizeofthing = sizeof(ui16) + imageRef[i]->name.length() + imageRef[i]->path.length() + ((ui8)imageRef[i]->tb);
				f.write((const char*)&sizeofthing, 2);
				// this is for just in case the engine doesn't support this type of object yet
				
				
				// the size of the name and the name itself
				sizeofthing = imageRef[i]->name.length();
				f.write((const char*)&sizeofthing, 2);
				f << imageRef[i]->name.c_str();
				
				// the size of the filepath and the filepath itself
				sizeofthing = imageRef[i]->path.length();
				f.write((const char*)&sizeofthing, 2);
				f << imageRef[i]->path.c_str();
				
				// the texture blend mode
				f.write((const char*)&imageRef[i]->tb, 1);
				
			}
		}
		
		/* ********************************************************************************
		 WRITE TEXTURES AND COLORS
		 ******************************************************************************* */
		if (textureRef.size() > 0)
		{
			for (int i = 0; i < textureRef.size(); i++)
			{
				// what kind of asset is this? (2 bytes)
				//f.write((const char*) 2, 2);
				f.put((const char) 2); // 2 = texture
				
				// what number asset are we at? (2 bytes)
				f.write((const char*) &i, 2);
				
				// what is the size of this asset? (2 bytes)
				ui16 sizeofthing = (sizeof(ui16) + textureRef[i]->name.length() + (sizeof(ui16) * 5));
				f.write((const char*)&sizeofthing, 2);
				// this is for just in case the engine doesn't support this type of object yet
				
				
				// the size of the name and the name itself
				sizeofthing = textureRef[i]->name.length();
				f.write((const char*)&sizeofthing, 2);
				f << textureRef[i]->name.c_str();
				
				// find the image this texture is linking to
				if (imageRef.size() > 0)
				{
					for (int j = 0; j < imageRef.size(); j++)
					{
						// theres got to be a faster way to do this
						if (imageRef[j] == textureRef[i]->getSource())
						{
							// what image does this texture use? (2 bytes)
							f.write((const char*) &j, 2);
							break;
						}
					}
				}
				
				// write the x and y offset into the file (2 + 2 bytes)
				static int tempx, tempy;
				
				textureRef[i]->getOffset(tempx, tempy);
				f.write((const char*) &tempx, 2);
				f.write((const char*) &tempy, 2);
				
				// write the x and y size into the file (2 + 2 bytes)
				textureRef[i]->getSize(tempx, tempy);
				f.write((const char*) &tempx, 2);
				f.write((const char*) &tempy, 2);
				
				// write the base color
				f.write((const char*)&textureRef[i]->r, 1);
				f.write((const char*)&textureRef[i]->g, 1);
				f.write((const char*)&textureRef[i]->b, 1);
				f.write((const char*)&textureRef[i]->a, 1);
			}
		}
		
		// terminate the asset list, move on to nodes
		//f.write((const char*) -1, 2);
		f.put((const char) -1);
		// -1 is used to tell the engine to move on to other stuff
		
		/* ********************************************************************************
		 WRITE ENTITIES
		 ******************************************************************************* */
		if (entityRef.size() > 0)
		{
			for (int i = 0; i < entityRef.size(); i++)
			{
			
				// put it in!
				//f.write((const char*) 1, 2);
				f.put((const char) 1);
				// 1 = entity
				
				// what number node is this?
				f.write((const char*) &i, 2);
				
				// size of the entity entry
				ui16 sizeofthing = entityRef[i]->name.length() + 26;
				f.write((const char*)&sizeofthing, 2);
				
				
				// name and size of name
				sizeofthing = entityRef[i]->name.length();
				f.write((const char*)&sizeofthing, 2);
				f << entityRef[i]->name.c_str();
				
				// render object
				// currently only supports the sprite class
				//f.write((const char*) 0, 2);
				f.put((const char) 0);
				
				// but is it drawable?
				f.write((const char*) &entityRef[i]->draw, 1);
				
				// color
				for (int j = 0; j < colorRef.size(); j++)
				{
					if (colorRef[j] == entityRef[i]->col)
					{
						f.write((const char*) &j, 2);
						break;
					}
				}
				
				float position[9] = {
					(float)entityRef[i]->pos.x,
					(float)entityRef[i]->pos.y,
					(float)entityRef[i]->pos.z,
					(float)entityRef[i]->rot.x,
					(float)entityRef[i]->rot.y,
					(float)entityRef[i]->rot.z,
					(float)entityRef[i]->scl.x,
					(float)entityRef[i]->scl.y,
					(float)entityRef[i]->scl.z
				};
				
				// entity position
				f.write((const char*) &position[0], sizeof(float));
				f.write((const char*) &position[1], sizeof(float));
				f.write((const char*) &position[2], sizeof(float));
				
				// entity rotation
				f.write((const char*) &position[3], sizeof(float));
				f.write((const char*) &position[4], sizeof(float));
				f.write((const char*) &position[5], sizeof(float));
				
				// entity scale
				f.write((const char*) &position[6], sizeof(float));
				f.write((const char*) &position[7], sizeof(float));
				f.write((const char*) &position[8], sizeof(float));
				
				// find the parent!
				if (entityRef[i]->parent != Tree->currentScene)
				{
					for (int j = 0; j < nodeRef.size(); j++)
					{
						if (entityRef[i]->parent == nodeRef[j])
						{
							f.write((const char*) &j, 2);
						}
					}
					
				} else {
					
					//f.write((const char*) -2, 2);
					f.put((const char) -2);
					// -2 is essentially a null pointer
					// because -1 is "move on" for the loader
				}
				
			}
		}
		
		/* ********************************************************************************
		 WRITE GRIDS
		 ******************************************************************************* */
		if (gridRef.size() > 0)
		{
			for (int i = 0; i < gridRef.size(); i++)
			{
				// put it in!
				//f.write((const char*) 2, 2);
				f.put((const char) 2);
				// 2 = grid
				
				// what number node is this?
				f.write((const char*) &i, 2);
				
				// size of the grid entry
				ui16 sizeofthing = gridRef[i]->name.length() + 30 + ((gridRef[i]->getX() * gridRef[i]->getY()) * 2);
				f.write((const char*)&sizeofthing, 2);
				
				// name and size of name
				sizeofthing = gridRef[i]->name.length();
				f.write((const char*)&sizeofthing, 2);
				f << gridRef[i]->name.c_str();
				
				// render object
				// currently only supports the sprite class
				//f.write((const char*) 0, 2);
				f.put((const char) 0);
				
				// but is it drawable?
				f.write((const char*) &gridRef[i]->draw, 1);
				
				// color
				// we don't need it for the whole grid
				
				// write grid size
				ui16 tempint = gridRef[i]->getX();
				f.write(reinterpret_cast<const char*>(&tempint), 2);
				tempint = gridRef[i]->getY();
				f.write(reinterpret_cast<const char*>(&tempint), 2);
				
				// go through the attributes of the tiles
				for (int j = 0; j < gridRef[i]->getX(); j++)
				{
					for (int k = 0; k < gridRef[i]->getY(); k++)
					{
						static bool tempbool = gridRef[i]->getTile(j, k)->getSolid();
						f.write((const char*) &tempbool, 1);
						
						for (int l = 0; l < textureRef.size(); l++)
						{
							if (gridRef[i]->getTile(j, k)->getTex() == textureRef[l])
							{
								f.write((const char*) &l, 2);
								break;
								// loop-ception!
							}
						}
					}
				}
				
				// grid position
				f.write((const char*) &gridRef[i]->pos.x, 4);
				f.write((const char*) &gridRef[i]->pos.y, 4);
				f.write((const char*) &gridRef[i]->pos.z, 4);
				
				// grid rotation
				f.write((const char*) &gridRef[i]->rot.x, 4);
				f.write((const char*) &gridRef[i]->rot.y, 4);
				f.write((const char*) &gridRef[i]->rot.z, 4);
				
				// grid scale
				f.write((const char*) &gridRef[i]->scl.x, 4);
				f.write((const char*) &gridRef[i]->scl.y, 4);
				f.write((const char*) &gridRef[i]->scl.z, 4);
				
				// find the parent!
				if (gridRef[i]->parent != nullptr)
				{
					for (int j = 0; j < nodeRef.size(); j++)
					{
						if (gridRef[i]->parent == nodeRef[j])
						{
							f.write((const char*) &j, 2);
						}
					}
					
				} else {
					
					//f.write((const char*) -2, 2);
					f.put((const char) -2);
					// -2 is essentially a null pointer
					// because -1 is "move on" for the loader
				}
			}
		}
		
		f.close();
		return 0;
	}
	
	int scene::readFromFile(std::string file)
	{
		std::string wFileExt = file;
		if (!wFileExt.find_last_of("."))
		{
			// no file extention. put one on
			wFileExt.append(".sinncascene");
		}
		
		std::ifstream f(wFileExt, std::ios::binary);
		if (!f.good())
		{
			printf("File stream is not good.");
			return -1;
			
		} else if (!f.is_open())
		{
			printf("Could not open %s", wFileExt.c_str());
			return -1;
		}
		/*
		std::streampos pos = f.tellg();
		char* buffer = (char*)Heap->allocate(pos, __alignof(char*));
		
		// read the file
		f.seekg(0, std::ios::beg);
		f.read(buffer, pos);
		f.close();
		// now we're done with the file and we can read from the buffer
		*/
		char buffer[8];
		ui16 secondaryHeader = 0;
		f.read(buffer, 8);
		f.read((char*)&secondaryHeader, 2);
		
		if (!strcmp(buffer, "sinnca_") && secondaryHeader != 18420)
		{
			f.close();
			printf("This is not a valid sinnca scene file");
			return -1;
		}
		
		ui32 noOfAssets = 0;
		f.read((char*)&noOfAssets, 4);
		ui32 noOfNodes = 0;
		f.read((char*)&noOfNodes, 4);
		
		for (int i = 0; i < noOfAssets; i++)
		{
			int obType = 0, numAsset = 0, assetSize = 0;
			
			f.read((char*)&obType, 1);
			f.read((char*)&numAsset, 2);
			f.read((char*)&assetSize, 2);
			
			switch (obType)
			{
				case 1:
				{
					// It's an image!
					int nameSize = 0;
					f.read((char*)&nameSize, 2);
					
					std::string thingName;
					for (int j = 0; j < nameSize; j++)
					{
						static char c;
						f.read(&c, 1);
						thingName.append(&c);
					}
					
					
					// how long is the path?
					int pathSize = 0;
					f.read((char*)&pathSize, 2);
					// and the path itself
					std::string pathName;
					for (int j = 0; j < pathSize; j++)
					{
						static char c;
						f.read(&c, 1);
						pathName.append(&c);
					}
					
					
					ui8 blendMode = 0;
					f.read((char*)&blendMode, 1);
					
					
					image* im = createImage(thingName);
					im->load(pathName, (texBlend)blendMode);
					Tree->currentScene->imageRef.push_back(im);
					
					break;
				}
				case 2:
				{
					// It's a texture!
					// how long is the name?
					int nameSize = 0;
					f.read((char*)&nameSize, 2);
					// and the name itself
					std::string thingName;
					for (int j = 0; j < nameSize; j++)
					{
						static char c;
						f.read(&c, 1);
						thingName.append(&c);
					}
					
					int imageLink = 0;
					f.read((char*)&imageLink, 2);
					
					// in-image offset
					int xOff = 0, yOff = 0;
					f.read((char*)&xOff, 2);
					f.read((char*)&yOff, 2);
					
					// texture dimentions
					int xSize = 0, ySize = 0;
					f.read((char*)&xSize, 2);
					f.read((char*)&ySize, 2);
					
					// base color
					ui8 col[4] = {0, 0, 0, 0};
					f.read((char*)&col[0], 1);
					f.read((char*)&col[1], 1);
					f.read((char*)&col[2], 1);
					f.read((char*)&col[3], 1);
					
					texture* tx = createTexture(thingName);
					tx->setSource(Tree->currentScene->imageRef[imageLink]);
					tx->setOffset(xOff, yOff);
					tx->setSize(xSize, ySize);
					
					tx->r = col[0];
					tx->g = col[1];
					tx->b = col[2];
					tx->a = col[3];
					
					break;
				}
				default:
				{
					int nameSize = 0;
					f.read((char*)&nameSize, 2);
					
					std::string thingName;
					f.read((char*)&thingName, nameSize);
					printf("Object '%s' was not recognized by this version of the engine. A newer version may be required or there was a mis-write when the file was created", thingName.c_str());
					
					break;
				}
			}
		}
		
		int tempint = 0;
		f.read((char*)&tempint, 1);
		
		for (int i = 0; i < noOfNodes; i++)
		{
			int obType = 0, numNode = 0, nodeSize = 0;
			
			f.read((char*)&obType, 1);
			f.read((char*)&numNode, 2);
			f.read((char*)&nodeSize, 2);
			
			switch (obType)
			{
				case 1:
				{
					// It's an entity!
					int nameSize = 0;
					f.read((char*)&nameSize, 2);
					
					std::string thingName;
					for (int j = 0; j < nameSize; j++)
					{
						static char c;
						f.read(&c, 1);
						thingName.append(&c);
					}
					
					// this space reserved for renderObj reading
					int renderLink = 0;
					f.read((char*)&renderLink, 1);
					
					// will this entity draw by default?
					bool willDraw = true;
					f.read((char*)&willDraw, 1);
					
					// Which texture will we use?
					int textureLink = 0;
					f.read((char*)&textureLink, 2);
					
					float position[9] = {0, 0, 0, 0, 0, 0, 0, 0, 0};
					for (int j = 0; j < 9 ; j++)
					{
						f.read((char*)&position[j], sizeof(float));
						//position[j] = endian32bit(position[j]);
					}
					
					
					int parentLink = -2;
					f.read((char*)&parentLink, 1);
					
					entity* en = createEntity(thingName);
					
					en->draw = willDraw;
					
					en->col = textureRef[textureLink];
					/*
					en->pos.x = position[0];
					en->pos.y = position[1];
					en->pos.z = position[2];
					en->rot.x = position[3];
					en->rot.y = position[4];
					en->rot.z = position[5];
					en->scl.x = position[6];
					en->scl.y = position[7];
					en->scl.z = position[8];
					*/
					if (parentLink == -2 || parentLink < 0)
					{
						en->parent = this;
						
					} else {
						
						en->parent = nodeRef[parentLink];
					}
					
					break;
				}
				
				case 2:
				{
					
					break;
				}
				default:
				{
					break;
				}
			}
		}
		
		f.close();
		return 0;
	}
	

I know there are some weird characters being written, but they were for safeguards I thought I needed at the time, and I really thought I was reading the file correctly.

Macbook Pro 2.66Ghz dual core, 4GB ram, 512MB vram, MacOSX 9.1, Windows 8.1
Xcode 5.0.2, C++, lua

This topic is closed to new replies.

Advertisement