Jump to content

  • Log In with Google      Sign In   
  • Create Account


#ActualAllEightUp

Posted 17 August 2013 - 12:47 PM

Much of your worry is what I call overkill.  There is a greatly simplified method of dealing with this which is not only faster for the actual read/write but also for the implementation.  My approach has always been to simply write the data files in the natural endian for the platform it is written from, so there is no dealing with endian at all during writes (beyond testing) and only deal with it during reads.  In order to make this work, you simply have an upfront file header which you read in which contains an endian flag.  Now, when a platform of a different endian opens a file it checks that bit and creates the appropriate read object which encapsulates all the endian aware reading.  This makes it such that 99.9% of the time you are working in the proper endian for the machine and don't have to deal with the slower reconstruction of always being endian aware.

 

Now, the code you post is a problem, let's fix that up:

buffer& operator<<(buffer& os, unsigned short x)
{
  uint8_t  part0 = uint8_t( x & 0xFF );
 uint8_t   part1 = uint8_t( (x & 0xFF00) >> 8 );
 os.put( part0 );
 os.put( part1 );
    return os;
};
 
// by side effect this works also:
buffer& operator << ( buffer& os, signed short x )
{
 unsigned short xx;
memcpy( &xx, &x, sizeof( signed short ) );  // Avoid aliasing, could use pointer tricks on most compilers though.
  os << xx;
 return os;
}

By using the extract and shift operations you don't care about the signed/unsigned nature of the data as per byte the data is identical and only the order of memory matters when reconstructing a complete value.

 

For floating point values, so long as the machines support IEEE form floats/doubles (pretty much everything out there), you can do exactly the same thing as the above just extend to 32/64 bits and use a temporary 'uint32_t' as the target of the memcpy.  Again, the only difference in such a case is byte order and you don't have to manipulate the bits in any manner.

 

NOTE:  Should have really written the code as:

buffer& operator<<(buffer& os, unsigned short x)
{
  os.write( (const char*)&x, sizeof( unsigned short ) );
  return os;
};

For the write side, just don't care about endian or dealing with the multibyte nature of the data.  The reading portion is where you would do the appropriate shifts or swaps if the endian needs to change.


#2AllEightUp

Posted 17 August 2013 - 12:47 PM

Much of your worry is what I call overkill.  There is a greatly simplified method of dealing with this which is not only faster for the actual read/write but also for the implementation.  My approach has always been to simply write the data files in the natural endian for the platform it is written from, so there is no dealing with endian at all during writes (beyond testing) and only deal with it during reads.  In order to make this work, you simply have an upfront file header which you read in which contains an endian flag.  Now, when a platform of a different endian opens a file it checks that bit and creates the appropriate read object which encapsulates all the endian aware reading.  This makes it such that 99.9% of the time you are working in the proper endian for the machine and don't have to deal with the slower reconstruction of always being endian aware.

 

Now, the code you post is a problem, let's fix that up:

buffer& operator<<(buffer& os, unsigned short x)
{
  uint8_t  part0 = uint8_t( x & 0xFF );
 uint8_t   part1 = uint8_t( (x & 0xFF00) >> 8 );
 os.put( part0 );
 os.put( part1 );
    return os;
};
 
// by side effect this works also:
buffer& operator << ( buffer& os, signed short x )
{
 unsigned short xx;
memcpy( &xx, &x, sizeof( signed short ) );  // Avoid aliasing, could use pointer tricks on most compilers though.
  os << xx;
 return os;
}

By using the extract and shift operations you don't care about the signed/unsigned nature of the data as per byte the data is identical and only the order of memory matters when reconstructing a complete value.

 

For floating point values, so long as the machines support IEEE form floats/doubles (pretty much everything out there), you can do exactly the same thing as the above just extend to 32/64 bits and use a temporary 'uint32_t' as the target of the memcpy.  Again, the only difference in such a case is byte order and you don't have to manipulate the bits in any manner.

 

NOTE:  Should have really written the code as:

buffer& operator<<(buffer& os, unsigned short x)
{
  os.write( &x, sizeof( unsigned short ) );
  return os;
};

For the write side, just don't care about endian or dealing with the multibyte nature of the data.  The reading portion is where you would do the appropriate shifts or swaps if the endian needs to change.


#1AllEightUp

Posted 17 August 2013 - 12:36 PM

Much of your worry is what I call overkill.  There is a greatly simplified method of dealing with this which is not only faster for the actual read/write but also for the implementation.  My approach has always been to simply write the data files in the natural endian for the platform it is written from, so there is no dealing with endian at all during writes (beyond testing) and only deal with it during reads.  In order to make this work, you simply have an upfront file header which you read in which contains an endian flag.  Now, when a platform of a different endian opens a file it checks that bit and creates the appropriate read object which encapsulates all the endian aware reading.  This makes it such that 99.9% of the time you are working in the proper endian for the machine and don't have to deal with the slower reconstruction of always being endian aware.

 

Now, the code you post is a problem, let's fix that up:

 

buffer& operator<<(buffer& os, unsigned short x)
{
  uint8_t  part0 = uint8_t( x & 0xFF );
 uint8_t   part1 = uint8_t( (x & 0xFF00) >> 8 );
 os.put( part0 );
 os.put( part1 );
    return os;
};
 
// by side effect this works also:
buffer& operator << ( buffer& os, signed short x )
{
 unsigned short xx;
memcpy( &xx, &x, sizeof( signed short ) );  // Avoid aliasing, could use pointer tricks on most compilers though.
  os << xx;
 return os;
}

 

By using the extract and shift operations you don't care about the signed/unsigned nature of the data as per byte the data is identical and only the order of memory matters when reconstructing a complete value.

 

For floating point values, so long as the machines support IEEE form floats/doubles (pretty much everything out there), you can do exactly the same thing as the above just extend to 32/64 bits and use a temporary 'uint32_t' as the target of the memcpy.  Again, the only difference in such a case is byte order and you don't have to manipulate the bits in any manner.


PARTNERS