• Advertisement

Archived

This topic is now archived and is closed to further replies.

endianness

This topic is 5296 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I''m making my game cross platform and I''m trying to understand this issue of endianness. Let''s say I compile and execute this: ofstream file("out", ios::out|ios::binary); int x = 1; file.write((char*)&x, 4); If I opened up out in visual studio, the file "out" would look like: 01 00 00 00 Let''s say I did the same with a Mac, or some big endian machine... Would the file "out" contain: 00 00 00 01 Basically just the bytes in reverse order? What about types like doubles and floats. I read and write those out to file, but I''m not even sure how they represent the numbers they do in the first place. How would I account for endianness for those types? Thanks - Andrew

Share this post


Link to post
Share on other sites
Advertisement
The way it''s solved for networking is a set of macros, which should be included with your compiler (it''s part of WinSock).

ntohl - Converts a long from network standard to host standard
ntohs - Converts a short from network standard to host standard
htonl - Converts a long from host standard to network standard
htons - Converts a short from host standard to network standard

Every value you want to read or write that isn''t a char must pass through these macros. Floats can be treated as a long, just be careful you''re not casting the float to a long as you''ll lose data (cast it''s pointer, or use a union, or some other method).

It''s a bit of a pain, but it''s isolated to one part of your code (file io), and it''s a well known standard.

Share this post


Link to post
Share on other sites
Thanks for the help, I'm sorry I wasn't specific though, I'm trying to understand this with respect to file IO, not WinSock.

[edited by - atcdevil on August 19, 2003 11:00:52 AM]

Share this post


Link to post
Share on other sites
Another question to add:

What about arrays:

If I have char a[] = "Hello";

Is there a difference (to me as the programmer) as to how "Hello" is stored in memory on machines differing in endianness?

Share this post


Link to post
Share on other sites
Yes, I know you wanted file io, but I''m saying the exact thing you''re trying to do has a standard solution in the networking world. Why not use their solution? In both cases it''s just preparing data for transfer, whether it''s as a packet or to disk doesn''t matter.

Basically every platform in existance already has the endian conversion code written to support networking. Instead of coming up with your own method, use the existing code knowing it will be pre-written, and bug free.

Share this post


Link to post
Share on other sites
Char arrays are fine as is (usually. There are some DSPs I''ve used where this won''t be the case, but you can safely ignore them).

Short arrays, and long arrays will need you to htons() and htonl() each value.

Share this post


Link to post
Share on other sites
Damn, I'm slow...

Thats basically it for 99% of the machines that you are likely to come across.

However actually making this fully portable sucks. Not only do you have trouble with big-endian vs little endian but there are such things as middle-endian where the byte order is something else.

There is also the problem with formats. Unsigned integers are fine as long as they are the same size, but I know of at least 3 different forms of signed interger (although I've never actually seen a machine that uses anything other than 2s complement). Floats and doubles may work if both the machines use IEEE standard (754-1985) but not all machines use this. (The only reference I have lying around for the format is here
[URL]http://laguerre.psc.edu/general/software/packages/ieee/ieee.html[/url])

If you can't get away with just using values of type uint8_t (unsigned char, if you don't have a C99 compiler) you can probably use the functions htonl, ntohl, htons, ntohs to portably store unsigned 16 and 32 bit values. Anything else starts getting a bit iffy.

[edited by - Robot guy on August 19, 2003 11:11:36 AM]

Share this post


Link to post
Share on other sites
I''ve always wondered, how does byte order affect 64-bit integers on 32-bit platforms? Are they stored as two "inverted" 4-byte values, or one 8-byte inverted value?

In other words, is it stored:

32107654

or

76543210

Share this post


Link to post
Share on other sites


//******************************************************

// $$C sBuffer

//

// General linear buffer manupulation. It bears no

// storage (uses external space), but keeps track of

// the data position and size in the buffer.

//

//******************************************************

class sBuffer
{
public:
//---- General

sBuffer( unsigned char * );
sBuffer( char * );
sBuffer( void * );
~sBuffer( );
//---- Encode.

void WriteByte( unsigned char );
void WriteShort( short );
void WriteLong( long );
void WriteFloat( float );
void WriteString( char * );
void WriteMulti( void *, int );
//---- Decode.

unsigned char ReadByte( void );
short ReadShort( void );
long ReadLong( void );
float ReadFloat( void );
char * ReadString( void );
void ReadMulti( void *, int );
public:
unsigned char * m_pData;
int m_iPos;
};


inline sBuffer::sBuffer( unsigned char * in_str )
: m_pData( in_str )
, m_iPos(0)
{
}
inline sBuffer::sBuffer( char * in_str )
: m_pData( (unsigned char*) in_str )
, m_iPos(0)
{
}
inline sBuffer::sBuffer( void * in_str )
: m_pData( (unsigned char*) in_str )
, m_iPos(0)
{
}

inline sBuffer::~sBuffer()
{
}

inline void sBuffer::WriteByte( unsigned char in_ch )
{
m_pData[ m_iPos++ ] = in_ch;
}

inline void sBuffer::WriteShort( short in_s )
{
m_pData[ m_iPos++ ] = (unsigned char) ( (in_s >> 8 ) & 0xFF );
m_pData[ m_iPos++ ] = (unsigned char) ( (in_s ) & 0xFF );
}

inline void sBuffer::WriteLong( long in_l )
{
m_pData[ m_iPos++ ] = (unsigned char) ( (in_l >> 24 ) & 0xFF );
m_pData[ m_iPos++ ] = (unsigned char) ( (in_l >> 16 ) & 0xFF );
m_pData[ m_iPos++ ] = (unsigned char) ( (in_l >> 8 ) & 0xFF );
m_pData[ m_iPos++ ] = (unsigned char) ( (in_l ) & 0xFF );
}

inline void sBuffer::WriteFloat( float in_f )
{
m_pData[ m_iPos++ ] = ( (unsigned char*)&in_f )[ 0 ];
m_pData[ m_iPos++ ] = ( (unsigned char*)&in_f )[ 1 ];
m_pData[ m_iPos++ ] = ( (unsigned char*)&in_f )[ 2 ];
m_pData[ m_iPos++ ] = ( (unsigned char*)&in_f )[ 3 ];
}

inline void sBuffer::WriteString( char * in_str )
{
while ( *in_str )
m_pData[ m_iPos++ ] = (unsigned char) *in_str++;
m_pData[ m_iPos++ ] = 0;
}

inline void sBuffer::WriteMulti( void * in_pData, int in_iLen )
{
::memmove( &m_pData[ m_iPos ], in_pData, in_iLen );
m_iPos += in_iLen;
}

inline unsigned char sBuffer::ReadByte( void )
{
return( m_pData[ m_iPos++ ] );
}

inline short sBuffer::ReadShort( void )
{
short s;
s = (short) m_pData[ m_iPos++ ];
s = (s << 8) + (short) m_pData[ m_iPos++ ];
return( s );
}

inline long sBuffer::ReadLong( void )
{
long l;
l = (long) m_pData[ m_iPos++ ];
l = (l << 8) + (long) m_pData[ m_iPos++ ];
l = (l << 8) + (long) m_pData[ m_iPos++ ];
l = (l << 8) + (long) m_pData[ m_iPos++ ];
return( l );
}

inline float sBuffer::ReadFloat( void )
{
float f;
( (unsigned char*)&f )[ 0 ] = m_pData[ m_iPos++ ];
( (unsigned char*)&f )[ 1 ] = m_pData[ m_iPos++ ];
( (unsigned char*)&f )[ 2 ] = m_pData[ m_iPos++ ];
( (unsigned char*)&f )[ 3 ] = m_pData[ m_iPos++ ];
return( f );
}

inline char * sBuffer::ReadString( void )
{
char * l_pStr = (char*) &m_pData[ m_iPos ];
while( m_pData[ m_iPos ] )
m_iPos ++;
m_iPos ++;
return( l_pStr );
}

inline void sBuffer::ReadMulti( void * in_pData, int in_iLen )
{
::memmove( in_pData, &m_pData[ m_iPos ], in_iLen );
m_iPos += in_iLen;
}



Writing:


char l_cHolder[ 4096 ];
sBuffer l_sBuff( l_cHolder );

l_sBuff.WriteShort( 12345 );
l_sBuff.WriteFloat( 3.1416 );
::write_to_file( ....., l_sBuff.m_pData, l_sBuff.m_iPos );


Reading:


void * incomingdata;
sBuffer l_sBuff( incomingdata );

short aaa = l_sBuff.ReadShort();
float bbb = l_sBuff.ReadFloat();


-cb




Share this post


Link to post
Share on other sites
Hello atcdevil,

for floats and double swapping should be the same as for int (32bit) and long (64bit). since the storage is the same.

I work with code that has to work on Intel/DEC little endian and work on Sun big endian.

I would advise that you pick one way to right data to file either little or big. Then of the system reading swap if different then file endianness(is this a word).

So if you save file as BIG when you read it in only swap if on a little endian system.

Also I belive on Intel (AMD) computers when write an int is store in a file is different then stored in memory.
basicly a int with a four bytes set as 1234 when save to file is 4321 I belive. Dose not realy matter since read will put it back in memory 1234. (am I right? tell me if I wrong I do most programming on Suns) Just thoght I throw that in.

Lord Bart

Share this post


Link to post
Share on other sites
Couldn''t you just write it out in network order and then read it back in and convert to host order?

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
quote:
Original post by atcdevil
Thanks for all the help everyone.

By the way, who thinks endian is a silly word??


It is derived from Gulliver''s Travels, which was a blatant political satire, so I guess "silly word" may be appropriate, as Jonothan Swift was probably attempting to show the absurdity of the Big-Endian and Little-Endian parties.

I haven''t read the book though.

Share this post


Link to post
Share on other sites

  • Advertisement