endianness
Author
I''m making my game cross platform and I''m trying to understand this issue of endianness.
Let''s say I compile and execute this:
ofstream file("out", ios::out|ios::binary);
int x = 1;
file.write((char*)&x, 4);
If I opened up out in visual studio, the file "out" would look like:
01 00 00 00
Let''s say I did the same with a Mac, or some big endian machine...
Would the file "out" contain:
00 00 00 01
Basically just the bytes in reverse order?
What about types like doubles and floats. I read and write those out to file, but I''m not even sure how they represent the numbers they do in the first place. How would I account for endianness for those types?
Thanks
- Andrew
The way it''s solved for networking is a set of macros, which should be included with your compiler (it''s part of WinSock).
ntohl - Converts a long from network standard to host standard
ntohs - Converts a short from network standard to host standard
htonl - Converts a long from host standard to network standard
htons - Converts a short from host standard to network standard
Every value you want to read or write that isn''t a char must pass through these macros. Floats can be treated as a long, just be careful you''re not casting the float to a long as you''ll lose data (cast it''s pointer, or use a union, or some other method).
It''s a bit of a pain, but it''s isolated to one part of your code (file io), and it''s a well known standard.
ntohl - Converts a long from network standard to host standard
ntohs - Converts a short from network standard to host standard
htonl - Converts a long from host standard to network standard
htons - Converts a short from host standard to network standard
Every value you want to read or write that isn''t a char must pass through these macros. Floats can be treated as a long, just be careful you''re not casting the float to a long as you''ll lose data (cast it''s pointer, or use a union, or some other method).
It''s a bit of a pain, but it''s isolated to one part of your code (file io), and it''s a well known standard.
Author
Thanks for the help, I'm sorry I wasn't specific though, I'm trying to understand this with respect to file IO, not WinSock.
[edited by - atcdevil on August 19, 2003 11:00:52 AM]
[edited by - atcdevil on August 19, 2003 11:00:52 AM]
Author
Another question to add:
What about arrays:
If I have char a[] = "Hello";
Is there a difference (to me as the programmer) as to how "Hello" is stored in memory on machines differing in endianness?
What about arrays:
If I have char a[] = "Hello";
Is there a difference (to me as the programmer) as to how "Hello" is stored in memory on machines differing in endianness?
Yes, I know you wanted file io, but I''m saying the exact thing you''re trying to do has a standard solution in the networking world. Why not use their solution? In both cases it''s just preparing data for transfer, whether it''s as a packet or to disk doesn''t matter.
Basically every platform in existance already has the endian conversion code written to support networking. Instead of coming up with your own method, use the existing code knowing it will be pre-written, and bug free.
Basically every platform in existance already has the endian conversion code written to support networking. Instead of coming up with your own method, use the existing code knowing it will be pre-written, and bug free.
Char arrays are fine as is (usually. There are some DSPs I''ve used where this won''t be the case, but you can safely ignore them).
Short arrays, and long arrays will need you to htons() and htonl() each value.
Short arrays, and long arrays will need you to htons() and htonl() each value.
Damn, I'm slow...
Thats basically it for 99% of the machines that you are likely to come across.
However actually making this fully portable sucks. Not only do you have trouble with big-endian vs little endian but there are such things as middle-endian where the byte order is something else.
There is also the problem with formats. Unsigned integers are fine as long as they are the same size, but I know of at least 3 different forms of signed interger (although I've never actually seen a machine that uses anything other than 2s complement). Floats and doubles may work if both the machines use IEEE standard (754-1985) but not all machines use this. (The only reference I have lying around for the format is here
http://laguerre.psc.edu/general/software/packages/ieee/ieee.html)
If you can't get away with just using values of type uint8_t (unsigned char, if you don't have a C99 compiler) you can probably use the functions htonl, ntohl, htons, ntohs to portably store unsigned 16 and 32 bit values. Anything else starts getting a bit iffy.
[edited by - Robot guy on August 19, 2003 11:11:36 AM]
Thats basically it for 99% of the machines that you are likely to come across.
However actually making this fully portable sucks. Not only do you have trouble with big-endian vs little endian but there are such things as middle-endian where the byte order is something else.
There is also the problem with formats. Unsigned integers are fine as long as they are the same size, but I know of at least 3 different forms of signed interger (although I've never actually seen a machine that uses anything other than 2s complement). Floats and doubles may work if both the machines use IEEE standard (754-1985) but not all machines use this. (The only reference I have lying around for the format is here
http://laguerre.psc.edu/general/software/packages/ieee/ieee.html)
If you can't get away with just using values of type uint8_t (unsigned char, if you don't have a C99 compiler) you can probably use the functions htonl, ntohl, htons, ntohs to portably store unsigned 16 and 32 bit values. Anything else starts getting a bit iffy.
[edited by - Robot guy on August 19, 2003 11:11:36 AM]
I''ve always wondered, how does byte order affect 64-bit integers on 32-bit platforms? Are they stored as two "inverted" 4-byte values, or one 8-byte inverted value?
In other words, is it stored:
32107654
or
76543210
In other words, is it stored:
32107654
or
76543210
//****************************************************** // $$C sBuffer // // General linear buffer manupulation. It bears no // storage (uses external space), but keeps track of // the data position and size in the buffer. // //******************************************************class sBuffer{ public: //---- General sBuffer( unsigned char * ); sBuffer( char * ); sBuffer( void * ); ~sBuffer( ); //---- Encode. void WriteByte( unsigned char ); void WriteShort( short ); void WriteLong( long ); void WriteFloat( float ); void WriteString( char * ); void WriteMulti( void *, int ); //---- Decode. unsigned char ReadByte( void ); short ReadShort( void ); long ReadLong( void ); float ReadFloat( void ); char * ReadString( void ); void ReadMulti( void *, int ); public: unsigned char * m_pData; int m_iPos;};inline sBuffer::sBuffer( unsigned char * in_str ) : m_pData( in_str ) , m_iPos(0){}inline sBuffer::sBuffer( char * in_str ) : m_pData( (unsigned char*) in_str ) , m_iPos(0){}inline sBuffer::sBuffer( void * in_str ) : m_pData( (unsigned char*) in_str ) , m_iPos(0){}inline sBuffer::~sBuffer(){}inline void sBuffer::WriteByte( unsigned char in_ch ){ m_pData[ m_iPos++ ] = in_ch;}inline void sBuffer::WriteShort( short in_s ){ m_pData[ m_iPos++ ] = (unsigned char) ( (in_s >> 8 ) & 0xFF ); m_pData[ m_iPos++ ] = (unsigned char) ( (in_s ) & 0xFF );}inline void sBuffer::WriteLong( long in_l ){ m_pData[ m_iPos++ ] = (unsigned char) ( (in_l >> 24 ) & 0xFF ); m_pData[ m_iPos++ ] = (unsigned char) ( (in_l >> 16 ) & 0xFF ); m_pData[ m_iPos++ ] = (unsigned char) ( (in_l >> 8 ) & 0xFF ); m_pData[ m_iPos++ ] = (unsigned char) ( (in_l ) & 0xFF );}inline void sBuffer::WriteFloat( float in_f ){ m_pData[ m_iPos++ ] = ( (unsigned char*)&in_f )[ 0 ]; m_pData[ m_iPos++ ] = ( (unsigned char*)&in_f )[ 1 ]; m_pData[ m_iPos++ ] = ( (unsigned char*)&in_f )[ 2 ]; m_pData[ m_iPos++ ] = ( (unsigned char*)&in_f )[ 3 ];}inline void sBuffer::WriteString( char * in_str ){ while ( *in_str ) m_pData[ m_iPos++ ] = (unsigned char) *in_str++; m_pData[ m_iPos++ ] = 0;}inline void sBuffer::WriteMulti( void * in_pData, int in_iLen ){ ::memmove( &m_pData[ m_iPos ], in_pData, in_iLen ); m_iPos += in_iLen;}inline unsigned char sBuffer::ReadByte( void ){ return( m_pData[ m_iPos++ ] );}inline short sBuffer::ReadShort( void ){ short s; s = (short) m_pData[ m_iPos++ ]; s = (s << 8) + (short) m_pData[ m_iPos++ ]; return( s );}inline long sBuffer::ReadLong( void ){ long l; l = (long) m_pData[ m_iPos++ ]; l = (l << 8) + (long) m_pData[ m_iPos++ ]; l = (l << 8) + (long) m_pData[ m_iPos++ ]; l = (l << 8) + (long) m_pData[ m_iPos++ ]; return( l );}inline float sBuffer::ReadFloat( void ){ float f; ( (unsigned char*)&f )[ 0 ] = m_pData[ m_iPos++ ]; ( (unsigned char*)&f )[ 1 ] = m_pData[ m_iPos++ ]; ( (unsigned char*)&f )[ 2 ] = m_pData[ m_iPos++ ]; ( (unsigned char*)&f )[ 3 ] = m_pData[ m_iPos++ ]; return( f );}inline char * sBuffer::ReadString( void ){ char * l_pStr = (char*) &m_pData[ m_iPos ]; while( m_pData[ m_iPos ] ) m_iPos ++; m_iPos ++; return( l_pStr );}inline void sBuffer::ReadMulti( void * in_pData, int in_iLen ){ ::memmove( in_pData, &m_pData[ m_iPos ], in_iLen ); m_iPos += in_iLen;}Writing:
char l_cHolder[ 4096 ];sBuffer l_sBuff( l_cHolder );l_sBuff.WriteShort( 12345 );l_sBuff.WriteFloat( 3.1416 );::write_to_file( ....., l_sBuff.m_pData, l_sBuff.m_iPos );Reading:
void * incomingdata;sBuffer l_sBuff( incomingdata );short aaa = l_sBuff.ReadShort();float bbb = l_sBuff.ReadFloat();-cb
Hello atcdevil,
for floats and double swapping should be the same as for int (32bit) and long (64bit). since the storage is the same.
I work with code that has to work on Intel/DEC little endian and work on Sun big endian.
I would advise that you pick one way to right data to file either little or big. Then of the system reading swap if different then file endianness(is this a word).
So if you save file as BIG when you read it in only swap if on a little endian system.
Also I belive on Intel (AMD) computers when write an int is store in a file is different then stored in memory.
basicly a int with a four bytes set as 1234 when save to file is 4321 I belive. Dose not realy matter since read will put it back in memory 1234. (am I right? tell me if I wrong I do most programming on Suns) Just thoght I throw that in.
Lord Bart
for floats and double swapping should be the same as for int (32bit) and long (64bit). since the storage is the same.
I work with code that has to work on Intel/DEC little endian and work on Sun big endian.
I would advise that you pick one way to right data to file either little or big. Then of the system reading swap if different then file endianness(is this a word).
So if you save file as BIG when you read it in only swap if on a little endian system.
Also I belive on Intel (AMD) computers when write an int is store in a file is different then stored in memory.
basicly a int with a four bytes set as 1234 when save to file is 4321 I belive. Dose not realy matter since read will put it back in memory 1234. (am I right? tell me if I wrong I do most programming on Suns) Just thoght I throw that in.
Lord Bart
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
