Reading from file to structs

Started by
21 comments, last by Hodgman 12 years, 6 months ago

Are you seriously suggesting a solution by abusing memory like that where you tell the compiler and user you have a one-character array for the string, and then storing the actual string content way outside the array and the object?

Yes. I am seriously suggesting that. I have a feeling you've seriously mis-understood what the code is doing.



That in itself is undefined,

No, it's very well defined. (Hint: Read the ISO standard on C strings)


and then your code doesn't even consider the fact that you're not aligning consecutive structures properly to ensure that their members are aligned.



Correct. But adding code to handle that is trival.



You cannot use your objects by themselves; they are only good for storing them as pointers in an array given your code to load them.

Correct. And that is bad because?


Your code will blow up as soon as you try do treat an object as a value.

That sir, is impossible. The compiler would inform you it can't be instanced long before that could possibly happen. I'm not *that* dumb ;)


What you propose is nothing more than a pointer and a dynamic sized string, but instead of having a safe implementation of the pointer, you're way into the realm of undefined behavior.

It is a safe implementation, and it is fully defined as per the ISO standard. You might not like it, but that's not something relevant to it's validity.

As for "What you propose is nothing more than a pointer and a dynamic sized string", well, look again. You may notice that there is no char pointer - which is the entire point of doing it in the first place! It is the most compact (and efficient) way of loading string data from a file stream.

I'd also gently point out that it's used all over the place, here are a couple of samples from the Win32 SDK:



typedef struct tagMETARECORD
{
DWORD rdSize;
WORD rdFunction;
WORD rdParm[1]; ///<<<<<<<<<<<
} METARECORD;


typedef struct _RGNDATA {
RGNDATAHEADER rdh;
char Buffer[1]; ///<<<<<<<<<<<
} RGNDATA, *PRGNDATA, NEAR *NPRGNDATA, FAR *LPRGNDATA;


And now you know about the technique, you'll probably notice it in most middleware libs too ;)

It's not the only technique that exists (there are other, equally nasty looking, but valid methods), but for the scenario above, you're going to struggle to find anything that can match it for performance, and the memory usage of the loaded asset.
Advertisement

[quote name='Tispe' timestamp='1318242643' post='4871030']The usual way of using in-place memory offsets:
[font="Courier New"]1) int offset = int((char*)(&object+1) - (char*)(&object.name)); //offset address is now between bar and baz?
2) object->name = ((char*)&object->name) + int(object->name);//did you mean offset?[/font]
1)Offset is the distance from the start of the 'name' field to the end of the structure. The string data itself is written to the file after the structure, so the offset tells you how far forward in the file to jump in order to find the string.
2) Upon deserialisation, 'name' actually contains the above offset value, not a pointer. The offset is relative to the address of the 'name' field, so the address of 'name' is added to the integer value of 'name', resulting in a pointer to the string data.
[/quote]

small problem:

struct LOADSPRITEOBJECT
{
char* name;
D3DCOLOR Color;
D3DXVECTOR3 OffsetPosition;
bool visible;
};


int offset = int((char*)(&object+1) - (char*)(&object.name));
write( &offset, sizeof(int) );


What happens is that (char*)(&object+1) - (char*)(&object.name) lands you exactly 3 bytes too far. This is because it assumes that there is only a (char*) written (1 Byte) when there is actually 4 bytes written (int). You must subract 3 bytes from offset.

The usual way of using in-place memory offsets:
struct Foo {
int bar;
char* name;
int baz;
};

-- serialization:
Foo object;
write( &object.bar, sizeof(int) );
int offset = int((char*)(&object+1) - (char*)(&object.name));
write( &offset, sizeof(int) );
write( &object.baz, sizeof(int) );
write( object.name, strlen(object.name)+1 );

-- deserialization:
void* buffer = readWholeFile();
Foo* object = (Foo*)buffer;
object->name = ((char*)&object->name) + int(object->name);


This is going to blow up right into your face the day you decide to port your code to 64bit. And your data file layout will change with the memory architecture. Ugh.
This is going to blow up right into your face the day you decide to port your code to 64bit. And your data file layout will change with the memory architecture. Ugh.
See the caveat in a later post regarding [font="'Courier New"]struct Offset[/font], which I find preferable to patching pointers into loaded blobs. If we do need to store pointers in blobs, I'd use the largest pointer type for the targets that the data file is being built for.
You must subract 3 bytes from offset.
Good catch. That's what I get for writing code from memory.

This topic is closed to new replies.

Advertisement