Sign in to follow this  
streamer

memory alignment

Recommended Posts

Hi all few months ago I had one issue (bug) when I was sending data over network. On some rare occasion it seemed that struct that I sent as message body didn't arrived all well. It had ints, chars etc. I resolved that changing struct to BYTE array, memcpy-ing bytes to int and everything was ok. I knew that it must be something with alignment of memory but I didn't know any better way to resolve the problem. Not so long ago I stumbled upon align command. Didn't even know that there is something like that. So easy: __declspec(align(1)) typedef struct and works. Question: What is the right way to align struct where you have several integers, and several bytes? Should I put align(sizeof(int)) or align(1)? Thanks in advance

Share this post


Link to post
Share on other sites
The compiler can rearrange and introduce padding in a struct in any way it sees fit. So, it is indeed quite unsafe to send that struct over the network or to a file.
Better than enforcing a specific memory alignment, you can tell the compiler not to shuffle the struct, but let it keep the layout you specified. You can do this by surrounding it with two pragmas:

#pragma pack(push,1)
struct MyStruct {
int a;
char b;
long c;
};
#pragma pack(pop)

Share this post


Link to post
Share on other sites
Of course, the #pragma solution begins to fall down as soon as you need your code to be compiled on more than one compiler (which may well be never), since #pragma directives are compiler specific and conforming compliers are required to silently ignore ones they don't support.

Be aware of this if multicompiler support is something you currently care about.

A more generalized solution is to implement some kind of serialization mechanism that will serialize instances of class types into a packed byte buffer. There are a number of ways to do this, from library solutions like Boost.Serialize, to writing it manually, to preprocessing and code generation tricks, et cetera.

Share this post


Link to post
Share on other sites
Thank guys for your replies.

So generally is it safe to:

#pragma pack(push,1)
struct MyStruct {
int a;
char b;
long c;
};
#pragma pack(pop)

if I work only on one compiler?

Share this post


Link to post
Share on other sites
If you are writing structs in binary format directly to files/sockets/etc., get in the habit of adding a compile-time assert to verify that the size is the size you expect.

e.g.:

struct MyStruct {
int a;
char b;
long c;
};
C_ASSERT(sizeof(MyStruct) == 9); // or whatever

Then if packing/alignment or other compiler optimizations kick in you'll know about at compile time.

C_ASSERT is a Windows thing. Boost also has BOOST_STATIC_ASSERT.

Share this post


Link to post
Share on other sites
Quote:
Original post by Anon Mike
If you are writing structs in binary format directly to files/sockets/etc., get in the habit of adding a compile-time assert to verify that the size is the size you expect.

e.g.:

struct MyStruct {
int a;
char b;
long c;
};
C_ASSERT(sizeof(MyStruct) == 9); // or whatever

Then if packing/alignment or other compiler optimizations kick in you'll know about at compile time.

C_ASSERT is a Windows thing. Boost also has BOOST_STATIC_ASSERT.


I have two bytes for crc check. [smile]

Share this post


Link to post
Share on other sites
Or, if you don't want to clutter your code with pragmas and such, you can fix the problem. Since you didn't say how you are actually receiving the structs, I can't really say what the problem was, except that it sounds strange. Did you send the struct by using sizeof(MyStruct) to get the struct's size? Or did you just sum up the component sizes by hand?

Share this post


Link to post
Share on other sites
Quote:
Original post by DaBono
The compiler can rearrange and introduce padding in a struct in any way it sees fit. So, it is indeed quite unsafe to send that struct over the network or to a file.
Better than enforcing a specific memory alignment, you can tell the compiler not to shuffle the struct, but let it keep the layout you specified. You can do this by surrounding it with two pragmas:
#pragma pack(push,1)
struct MyStruct {
int a;
char b;
long c;
};
#pragma pack(pop)
This is not quite true. A C compiler must lay out the members of a struct with memory address which increase in the order they are declared, with the exception of bitfield members. So in the struct:
struct MyStruct
{
int a;
char b;
long c;
};
a must always precede b, which must always precede c.

In C++ things are slightly more complicated. A C++ compiler must lay out the nonstatic data members of a class or struct which do not contain an intervening access-specifier with memory address which increase in the order thay are declared. So in the struct:
struct MyStruct
{
int a;
char b;
private:
long c;
float d;
public:
short e;
bool f;
};
a must always precede b, c must always precede d and e must always precede f, but no other guarantees can be made to the order.

In both C and C++ no guarantee is made about how much higher the address of a data member will be compared to the previous data member, with the exception of alignment requirements.

The pragmas you mentioned will do nothing to change the order in which the data members are laid out, but will guarantee that for two consecutive data members n and m the difference between the addresses of n and m will be the least multiple at least the size of n of the alignment you specify.

Σnigma

Share this post


Link to post
Share on other sites
Quote:
Original post by streamer
But if all members are public: then it shouldn't be the problem. Or?

You'll also need to avoid virtual inheritance and virtual functions, which I neglected to mention. But yes, in the absence of the above, with all non-static data members under a single access specifier, wrapped with pragmas on a compiler that supports them, on a platform with no alignment requirements you'll be fine.

Σnigma

Share this post


Link to post
Share on other sites
Quote:
Original post by Enigma
Quote:
Original post by streamer
But if all members are public: then it shouldn't be the problem. Or?

You'll also need to avoid virtual inheritance and virtual functions, which I neglected to mention. But yes, in the absence of the above, with all non-static data members under a single access specifier, wrapped with pragmas on a compiler that supports them, on a platform with no alignment requirements you'll be fine.

Σnigma


I guess then it will be fine [smile] it is under single access specifier, wrapped with pragmas, compiler supports pragmas, oooo and x86 platform have aligment requirements?

Share this post


Link to post
Share on other sites
Yes, you should be fine, providing you're not using SSE and aligned loads/stores. Also note that unaligned data access on x86 can result in reduced performance. It's not normally a problem since packed structures are only usually used for I/O, but be aware of it and don't just blindly use the same packed structures in your inner game loop!

Σnigma

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this