You may be better off doing more programming on a single machine, until you are more at ease with memory, bytes, how data structures are represented, and other such systems programming details.

Anyway, yes, a TCP stream of packets, and a file on disk, are actually very much similar. What my proposed marshaling does is define a particular struct per message type, and then build *one* template function per struct. All this function does is visit each struct member in order, calling into some template argument for each. You can then provide two classes: an "input reader" and an "output writer," which lets you read into the struct, or write out of the struct, depending on which you pass in.

What's extra elegant about this is that you can also write a class that does things like "build a GUI" or "send to log file" or whatever -- when you have the struct, and the visitor function, you can do many things to any instance of that struct, by simply building a different "stream" class, and using the same visitor. This allows you to separate the concerns of what the data is (the struct), how to describe what the data is (the visitor function), and what you do with a description of the data (the different class implementations).

This means you don't have to write separate "send()" and "receive()" and "editInGUI()" and "dumpAsXML()" and "logToFile()" functions for the same data struct, which is a pretty big win once your program becomes bigger.

@agisler
The C++ Middleware Writer -- http://webEbenezer.n...ntegration.html -- is an on line code generator that writes C++ marshalling code based on high-level user input. It automates the creation of functions that hplus has mentioned.


Brian Wood
Ebenezer Enterprises
http://webEbenezer.net

This is a bit over my head but I do not have a problem reading about it and trying to learn this. However what's the benefits of me doing it this way?

Just to add some of my own commentary in addition to the points hplus already mentioned:

To put simply, rather than having to write 4 pieces of logic: stream writer, stream reader, object writer, object reader, you will only have to write 3: stream writer, stream reader, object visitor. It might not seem like much at first, but as your project gets larger, the difference between the two methods' amount of code is huge. In addition, when you use the object writer and object reader approach, you have two pieces of logic you have to maintain and keep track up for maintenance whereas the object visitor is only one piece of logic.

By going the object visitor route, you further decrease development time when you have base types that can be reused. For example, let's say you have 5 different messages that all contain the same sequence of data, like entity id, X, Y, Z. In the object reader/writer approach, you will simply have logic to read/write each of those fields individually in all of your functions. In the object visitor approach, if you combined those 4 fields into a base type, you would only have to write the visit function logic once for that type, then reap the benefits of being able to reuse it for any message that uses it. So rather than having 8 lines of actual reading/writing code total, you only have 1. That is because the visitor pattern handles both reading and writing!

Here is a real world example. Consider the following structure that contains data about a security protocol:
[spoiler]

struct Data_Security
{
unsigned char mode;
unsigned __int64 initial_key;
unsigned int seed_count;
unsigned int crc_seed;
unsigned __int64 handshake_key;
unsigned int g;
unsigned int p;
unsigned int A;

Data_Security()
{
mode = 0;
initial_key = 0;
seed_count = 0;
crc_seed = 0;
handshake_key = 0;
g = 0;
p = 0;
A = 0;
}
};
[/spoiler]

Using the object writer/reader approach, we would have the following two functions:
[spoiler]
Data_Security FromStream( StreamReader stream )
{
Data_Security object;
object.mode = stream.ReadUInt8();
if( object.mode & 2 )
{
object.initial_key = stream.ReadUInt64();
}
if( object.mode & 4 )
{
object.seed_count = stream.ReadUInt32();
object.crc_seed = stream.ReadUInt32();
}
if( object.mode & 8 )
{
object.handshake_key = stream.ReadUInt64();
object.g = stream.ReadUInt32();
object.p = stream.ReadUInt32();
object.A = stream.ReadUInt32();
}
if( object.mode & 16 )
{
object.handshake_key = stream.ReadUInt64();
}
}

void ToStream( StreamWriter stream, const Data_Security & object )
{
stream.WriteUInt8( object.mode );
if( object.mode & 2 )
{
stream.WriteUInt64( object.initial_key );
}
if( object.mode & 4 )
{
stream.WriteUInt32( object.seed_count );
stream.WriteUInt32( object.crc_seed );
}
if( object.mode & 8 )
{
stream.WriteUInt64( object.handshake_key );
stream.WriteUInt32( object.g );
stream.WriteUInt32( object.p );
stream.WriteUInt32( object.A );
}
if( object.mode & 16 )
{
stream.WriteUInt64( object.handshake_key );
}
}
[/spoiler]

Where the WriteXXX / ReadXXX functions are coded as part of the StreamWriter / StreamReader class.

Now, for the object visitor pattern, we only have one function:
[spoiler]
template< typename Stream >
Stream & visit( Data_Security & value, Stream & stream )
{
stream.visit( "mode", value.mode );
if( value.mode & 2 )
{
stream.visit( "initial_key", value.initial_key );
}
if( value.mode & 4 )
{
stream.visit( "seed_count", value.seed_count );
stream.visit( "crc_seed", value.crc_seed );
}
if( value.mode & 8 )
{
stream.visit( "handshake_key", value.handshake_key );
stream.visit( "g", value.g );
stream.visit( "p", value.p );
stream.visit( "A", value.A );
}
if( value.mode & 16 )
{
stream.visit( "handshake_key", value.handshake_key );
}
return stream;
}
[/spoiler]

We still have both StreamReader and StreamWriter classes, but rather than them defining Read/Write named functions, they all use the same "visit" function with different logic depending on the object.

So we are taking advantage of the way C++ works to drastically cut down on the work and code needed to implement object serialization. The more types you have, the more visit functions you do have to write, but you only have to write them once, so you can easily reuse them in the future. As mentioned before, as your project grows, the object visitor pattern pays for itself.

The object reader/writer code shown is typically how you see people do it. I myself used that style for years because I was unaware of the visitor pattern. Now that I understand it better, I can see how beneficial it is and how there really is no reason to use the object reader/writer method because everything you can accomplish there, you can accomplish with the visitor pattern; you just might need to add a state object to know some extra information.

Here's some simple examples of more complete visitor stream classes shown in hplus's earlier post:
SeralizeStream
[spoiler]
class SeralizeStream
{
private:
std::vector< unsigned char > m_buffer;

public:
SeralizeStream & visit( const std::string & name, const std::string & value )
{
#ifdef _DEBUG
std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << std::endl;
#endif
size_t size = value.size();
do
{
if( size >= 255 )
{
m_buffer.push_back( 255 );
size -= 254;
}
else
{
m_buffer.push_back( static_cast< unsigned char >( size ) );
size -= size;
}
} while( size != 0 );
m_buffer.insert( m_buffer.end(), value.begin(), value.end() );
return *this;
}

SeralizeStream & visit( const std::string & name, const unsigned char & value )
{
#ifdef _DEBUG
std::cout << "[" << __FUNCTION__<< "] " << name << " => " << (int)value << " (0x" << std::hex << (int)value << std::dec << ")" << std::endl;
#endif
m_buffer.push_back( value );
return *this;
}

SeralizeStream & visit( const std::string & name, const signed char & value )
{
#ifdef _DEBUG
std::cout << "[" << __FUNCTION__<< "] " << name << " => " << (int)value << " (0x" << std::hex << (int)value << std::dec << ")" << std::endl;
#endif
m_buffer.push_back( value );
return *this;
}

SeralizeStream & visit( const std::string & name, const unsigned short & value )
{
#ifdef _DEBUG
std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << " (0x" << std::hex << value << std::dec << ")" << std::endl;
#endif
#if ENDIAN_BIG == 1
m_buffer.push_back( ( value >> 8 ) & 0xFF );
m_buffer.push_back( ( value >> 0 ) & 0xFF );
#elif ENDIAN_LITTLE == 1
m_buffer.push_back( ( value >> 0 ) & 0xFF );
m_buffer.push_back( ( value >> 8 ) & 0xFF );
#endif
return *this;
}

SeralizeStream & visit( const std::string & name, const signed short & value )
{
#ifdef _DEBUG
std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << " (0x" << std::hex << value << std::dec << ")" << std::endl;
#endif
#if ENDIAN_BIG == 1
m_buffer.push_back( ( value >> 8 ) & 0xFF );
m_buffer.push_back( ( value >> 0 ) & 0xFF );
#elif ENDIAN_LITTLE == 1
m_buffer.push_back( ( value >> 0 ) & 0xFF );
m_buffer.push_back( ( value >> 8 ) & 0xFF );
#endif
return *this;
}

SeralizeStream & visit( const std::string & name, const unsigned int & value )
{
#ifdef _DEBUG
std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << " (0x" << std::hex << value << std::dec << ")" << std::endl;
#endif
#if ENDIAN_BIG == 1
m_buffer.push_back( ( value >> 24 ) & 0xFF );
m_buffer.push_back( ( value >> 16 ) & 0xFF );
m_buffer.push_back( ( value >> 8 ) & 0xFF );
m_buffer.push_back( ( value >> 0 ) & 0xFF );
#elif ENDIAN_LITTLE == 1
m_buffer.push_back( ( value >> 0 ) & 0xFF );
m_buffer.push_back( ( value >> 8 ) & 0xFF );
m_buffer.push_back( ( value >> 16 ) & 0xFF );
m_buffer.push_back( ( value >> 24 ) & 0xFF );
#endif
return *this;
}

SeralizeStream & visit( const std::string & name, const signed int & value )
{
#ifdef _DEBUG
std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << " (0x" << std::hex << value << std::dec << ")" << std::endl;
#endif
#if ENDIAN_BIG == 1
m_buffer.push_back( ( value >> 24 ) & 0xFF );
m_buffer.push_back( ( value >> 16 ) & 0xFF );
m_buffer.push_back( ( value >> 8 ) & 0xFF );
m_buffer.push_back( ( value >> 0 ) & 0xFF );
#elif ENDIAN_LITTLE == 1
m_buffer.push_back( ( value >> 0 ) & 0xFF );
m_buffer.push_back( ( value >> 8 ) & 0xFF );
m_buffer.push_back( ( value >> 16 ) & 0xFF );
m_buffer.push_back( ( value >> 24 ) & 0xFF );
#endif
return *this;
}

SeralizeStream & visit( const std::string & name, const unsigned __int64 & value )
{
#ifdef _DEBUG
std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << " (0x" << std::hex << value << std::dec << ")" << std::endl;
#endif
#if ENDIAN_BIG == 1
m_buffer.push_back( ( value >> 56 ) & 0xFF );
m_buffer.push_back( ( value >> 48 ) & 0xFF );
m_buffer.push_back( ( value >> 40 ) & 0xFF );
m_buffer.push_back( ( value >> 32 ) & 0xFF );
m_buffer.push_back( ( value >> 24 ) & 0xFF );
m_buffer.push_back( ( value >> 16 ) & 0xFF );
m_buffer.push_back( ( value >> 8 ) & 0xFF );
m_buffer.push_back( ( value >> 0 ) & 0xFF );
#elif ENDIAN_LITTLE == 1
m_buffer.push_back( ( value >> 0 ) & 0xFF );
m_buffer.push_back( ( value >> 8 ) & 0xFF );
m_buffer.push_back( ( value >> 16 ) & 0xFF );
m_buffer.push_back( ( value >> 24 ) & 0xFF );
m_buffer.push_back( ( value >> 32 ) & 0xFF );
m_buffer.push_back( ( value >> 40 ) & 0xFF );
m_buffer.push_back( ( value >> 48 ) & 0xFF );
m_buffer.push_back( ( value >> 56 ) & 0xFF );
#endif
return *this;
}

SeralizeStream & visit( const std::string & name, const signed __int64 & value )
{
#ifdef _DEBUG
std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << " (0x" << std::hex << value << std::dec << ")" << std::endl;
#endif
#if ENDIAN_BIG == 1
m_buffer.push_back( ( value >> 56 ) & 0xFF );
m_buffer.push_back( ( value >> 48 ) & 0xFF );
m_buffer.push_back( ( value >> 40 ) & 0xFF );
m_buffer.push_back( ( value >> 32 ) & 0xFF );
m_buffer.push_back( ( value >> 24 ) & 0xFF );
m_buffer.push_back( ( value >> 16 ) & 0xFF );
m_buffer.push_back( ( value >> 8 ) & 0xFF );
m_buffer.push_back( ( value >> 0 ) & 0xFF );
#elif ENDIAN_LITTLE == 1
m_buffer.push_back( ( value >> 0 ) & 0xFF );
m_buffer.push_back( ( value >> 8 ) & 0xFF );
m_buffer.push_back( ( value >> 16 ) & 0xFF );
m_buffer.push_back( ( value >> 24 ) & 0xFF );
m_buffer.push_back( ( value >> 32 ) & 0xFF );
m_buffer.push_back( ( value >> 40 ) & 0xFF );
m_buffer.push_back( ( value >> 48 ) & 0xFF );
m_buffer.push_back( ( value >> 56 ) & 0xFF );
#endif
return *this;
}

SeralizeStream & visit( const std::string & name, const float & value )
{
#ifdef _DEBUG
std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << std::endl;
#endif
return visit( name, *( unsigned int * )( &value ) );
}

SeralizeStream & visit( const std::string & name, const double & value )
{
#ifdef _DEBUG
std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << std::endl;
#endif
return visit( name, *( unsigned __int64 * )( &value ) );
}

std::vector< unsigned char > & Buffer()
{
return m_buffer;
}

void Clear()
{
m_buffer.clear();
}
};
[/spoiler]

DeseralizeStream
[spoiler]
class DeseralizeStream
{
private:
std::vector<unsigned char> & m_buffer;
size_t m_index;

public:
DeseralizeStream(std::vector<unsigned char> & buffer)
: m_buffer( buffer ), m_index( 0 )
{
}

DeseralizeStream & visit( const std::string & name, std::string & value )
{
size_t size = 0;
while( m_buffer[ m_index ] == 255 )
{
size += 254;
++m_index;
}
size += m_buffer[ m_index++ ];
value.resize( size );
std::copy( m_buffer.begin() + m_index, m_buffer.begin() + m_index + size, value.begin() );
m_index += size;
#ifdef _DEBUG
std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << std::endl;
#endif
return *this;
}

DeseralizeStream & visit( const std::string & name, unsigned char & value )
{
value = (unsigned char)m_buffer[ m_index++ ];
#ifdef _DEBUG
std::cout << "[" << __FUNCTION__<< "] " << name << " => " << (int)value << " (0x" << std::hex << (int)value << std::dec << ")" << std::endl;
#endif
return *this;
}

DeseralizeStream & visit( const std::string & name, signed char & value )
{
value = (signed char)m_buffer[ m_index++ ];
#ifdef _DEBUG
std::cout << "[" << __FUNCTION__<< "] " << name << " => " << (int)value << " (0x" << std::hex << (int)value << std::dec << ")" << std::endl;
#endif
return *this;
}

DeseralizeStream & visit( const std::string & name, unsigned short & value )
{
#if ENDIAN_BIG == 1
value = (unsigned short)m_buffer[ m_index++ ] << 8 | (unsigned short)m_buffer[ m_index++ ] << 0;
#elif ENDIAN_LITTLE == 1
value = (unsigned short)m_buffer[ m_index++ ] << 0 | (unsigned short)m_buffer[ m_index++ ] << 8;
#endif
#ifdef _DEBUG
std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << " (0x" << std::hex << value << std::dec << ")" << std::endl;
#endif
return *this;
}

DeseralizeStream & visit( const std::string & name, signed short & value )
{
#if ENDIAN_BIG == 1
value = (signed short)m_buffer[ m_index++ ] << 8 | (signed short)m_buffer[ m_index++ ] << 0;
#elif ENDIAN_LITTLE == 1
value = (signed short)m_buffer[ m_index++ ] << 0 | (signed short)m_buffer[ m_index++ ] << 8;
#endif
#ifdef _DEBUG
std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << " (0x" << std::hex << value << std::dec << ")" << std::endl;
#endif
return *this;
}

DeseralizeStream & visit( const std::string & name, unsigned int & value )
{
#if ENDIAN_BIG == 1
value = (unsigned int)m_buffer[ m_index++ ] << 24 | (unsigned int)m_buffer[ m_index++ ] << 16 | (unsigned int)m_buffer[ m_index++ ] << 8 | (unsigned int)m_buffer[ m_index++ ] << 0;
#elif ENDIAN_LITTLE == 1
value = (unsigned int)m_buffer[ m_index++ ] << 0 | (unsigned int)m_buffer[ m_index++ ] << 8 | (unsigned int)m_buffer[ m_index++ ] << 16 | (unsigned int)m_buffer[ m_index++ ] << 24;
#endif
#ifdef _DEBUG
std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << " (0x" << std::hex << value << std::dec << ")" << std::endl;
#endif
return *this;
}

DeseralizeStream & visit( const std::string & name, signed int & value )
{
#if ENDIAN_BIG == 1
value = (signed int)m_buffer[ m_index++ ] << 24 | (signed int)m_buffer[ m_index++ ] << 16 | (signed int)m_buffer[ m_index++ ] << 8 | (signed int)m_buffer[ m_index++ ] << 0;
#elif ENDIAN_LITTLE == 1
value = (signed int)m_buffer[ m_index++ ] << 0 | (signed int)m_buffer[ m_index++ ] << 8 | (signed int)m_buffer[ m_index++ ] << 16 | (signed int)m_buffer[ m_index++ ] << 24;
#endif
#ifdef _DEBUG
std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << " (0x" << std::hex << value << std::dec << ")" << std::endl;
#endif
return *this;
}

DeseralizeStream & visit( const std::string & name, unsigned __int64 & value )
{
#if ENDIAN_BIG == 1
value = (unsigned __int64)m_buffer[ m_index++ ] << 56 | (unsigned __int64)m_buffer[ m_index++ ] << 48 | (unsigned __int64)m_buffer[ m_index++ ] << 40 | (unsigned __int64)m_buffer[ m_index++ ] << 32 | (unsigned __int64)m_buffer[ m_index++ ] << 24 | (unsigned __int64)m_buffer[ m_index++ ] << 16 | (unsigned __int64)m_buffer[ m_index++ ] << 8 | (unsigned __int64)m_buffer[ m_index++ ] << 0;
#elif ENDIAN_LITTLE == 1
value = (unsigned __int64)m_buffer[ m_index++ ] << 0 | (unsigned __int64)m_buffer[ m_index++ ] << 8 | (unsigned __int64)m_buffer[ m_index++ ] << 16 | (unsigned __int64)m_buffer[ m_index++ ] << 24 | (unsigned __int64)m_buffer[ m_index++ ] << 32 | (unsigned __int64)m_buffer[ m_index++ ] << 40 | (unsigned __int64)m_buffer[ m_index++ ] << 48 | (unsigned __int64)m_buffer[ m_index++ ] << 56;
#endif
#ifdef _DEBUG
std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << " (0x" << std::hex << value << std::dec << ")" << std::endl;
#endif
return *this;
}

DeseralizeStream & visit( const std::string & name, signed __int64 & value )
{
#if ENDIAN_BIG == 1
value = (signed __int64)m_buffer[ m_index++ ] << 56 | (signed __int64)m_buffer[ m_index++ ] << 48 | (signed __int64)m_buffer[ m_index++ ] << 40 | (signed __int64)m_buffer[ m_index++ ] << 32 | (signed __int64)m_buffer[ m_index++ ] << 24 | (signed __int64)m_buffer[ m_index++ ] << 16 | (signed __int64)m_buffer[ m_index++ ] << 8 | (signed __int64)m_buffer[ m_index++ ] << 0;
#elif ENDIAN_LITTLE == 1
value = (signed __int64)m_buffer[ m_index++ ] << 0 | (signed __int64)m_buffer[ m_index++ ] << 8 | (signed __int64)m_buffer[ m_index++ ] << 16 | (signed __int64)m_buffer[ m_index++ ] << 24 | (signed __int64)m_buffer[ m_index++ ] << 32 | (signed __int64)m_buffer[ m_index++ ] << 40 | (signed __int64)m_buffer[ m_index++ ] << 48 | (signed __int64)m_buffer[ m_index++ ] << 56;
#endif
#ifdef _DEBUG
std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << " (0x" << std::hex << value << std::dec << ")" << std::endl;
#endif
return *this;
}

DeseralizeStream & visit( const std::string & name, float & value )
{
unsigned int v;
visit( name, v );
memcpy( &value, &v, 4 );
#ifdef _DEBUG
std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << std::endl;
#endif
return *this;
}

DeseralizeStream & visit( const std::string & name, double & value )
{
unsigned __int64 v;
visit( name, v );
memcpy( &value, &v, 8 );
#ifdef _DEBUG
std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << std::endl;
#endif
return *this;
}
};
[/spoiler]

They are pretty basic classes. More functions for vectors, lists, maps, etc... could be added as needed. Also the way you work with strings might vary. Some protocols use fixed size strings, some use a variable length variable size type similar to the one shown, and others just use a variable length fixed size type (as shown in hplus's post with a 1 byte length limitation). I think the float/double logic is correct, but I might be wrong. Another thing to be careful of is ensuring you are using portable types (I'm not purposefully for the sake of a simple test). There are a few gotchas here you have to be careful of if you are going cross-platform or 32/64-bit different architectures. The most annoying one is the differences between wchar_t size on gcc on linux (4 bytes usually) and the size on windows (2 bytes usually). If you tried to send a string from one platform to the other without keeping this in mind, you can be in for some real headaches!(I.e. Windows Client <-> Linux Proxy Server <-> Windows Server).

Anyways, hopefully that adds to the useful information in this thread. Also as a disclaimer, all code was written during the course of reading this thread, so it might have bugs, do not use it without understanding what it does first. Good luck!

So we are taking advantage of the way C++ works to drastically cut down on the work and code needed to implement object serialization. The more types you have, the more visit functions you do have to write, but you only have to write them once, so you can easily reuse them in the future. As mentioned before, as your project grows, the object visitor pattern pays for itself.

Don't forget that you have to revisit the visit functions when you make changes to your types. With the C++ Middleware Writer you *don't* have to write or maintain visit functions.

Brian Wood
Ebenezer Enterprises
http://webEbenezer.net

But you have to specify it somewhere, right?

No. The CMW (C++ Middleware Writer) rereads updated header files and recreates the marshalling code as needed.

C++ lacks reflection. Plus if I remember correctly with your library you need to have code compiled by your server or something? That sounds far more complex to me than maintaining the visit functions.
[/quote]

You have to have to submit some code to the CMW, but I don't think it is complex. Getting set up to use the CMW takes about 15 minutes. That includes downloading the prerequisite Loki library and setting up an account. I use the following architecture:

CMW (server)
|
CMW Ambassador (server)
|
direct program (runs once and exits)

You download and build the CMW Ambassador and direct programs. The downloading and building takes less than 5 minutes. (Someone downloaded the software yesterday and reported having build problems. They hadn't downloaded the Loki library.)

Brian


There are two phases to setting up the CMW. I've described the first phase -- getting output from it -- the same output that comes in the archive. The second phase is beginning to use it in your project. The 15 minutes I mention is for getting the first phase working. The second phase though isn't difficult either.

I still don't get why you're trying to do this as a server. No sane developer would actually place the health of his project in the hands of a server operated by some random guy.
Even when "some random guy" is really big (say, IBM size) you can get screwed, because whoever operates the server may at some point:
1) get hit by a bus

Couldn't you throw in a "G-d forbid" here or there?

2) get hacked by some bad guys
3) decide that it's not profitable or fun
and you, as a developer, are cut off -- no more server available, and you can no longer build your project.

When you buy tools (rather than services,) you can keep running the tools for as long as you want.
Compare Blade3D -- a game development engine that was using a monthly payment/service model. At some point, they just decided to stop providing this service. Oops!
[/quote]

I'm sure you've heard of all the countries around the world that have over 20% unemployment. The official rate in the US is no longer really accurate -- http://www.investors...89-BLS-Rate.htm . Egypt, Spain, Greece -- it is a long list -- have very high unemployment. I provide the hosting for people around the world who can't afford to buy the service as a tool. I'm not really interested in marketing this to IBM and other well-known companies out there. I'm aiming for the little guy. There's no shortage of people who need help with their projects. Thanks to some lame excuses for "leaders", the world is in bad shape. Some people may not like my terms, but that doesn't change their situations. They like to eat and drink, have heat, buy clothes, buy their children toys... Anyway, I think its obvious that as I get more users my position becomes stronger -- I get better at preventing thieves (hacking), running the business and dodging buses.

The world is changing. I've long thought for example, that Microsoft is at a disadvantage to Google. Google wasn't afraid of the cloud; they embraced it. Microsoft goes around trying to get countries to crack down on software theft. I think that's a tough proposition and they are huge. I don't have to worry about things like that with this model.

Brian

WOW! I have not checked the thread in the last few days as I have been doing my homework on this subject

I think now I understand the basic concepts of hplus code. So I will just check with you that I have the right idea of the code.

The template is what allows you create different streams of any type. The class therefore tells you how to read in the different types of streams. Also the buf.push_back((i >> 24) & oxff); code this tells you where to start reading the data from.

A few questions regarding it as well.

what library is the std::vector contained in?

I am a bit uncertain about something as well. Is each InStream function reading a whole structure based on the type that comes in or just reading the individual data types that come in? Does that make sense?

@Drew_Benton

Thanks for the additional comments. The benefits are more clear now as well. I do understand for the most part the object visitor code you have shown me (not the real world example). From what I understand it works very similar to hplus code.

@wood_Brian

Thanks for the input. However as this is going to be a final year university project, so I need to keep everything local so that I can work on it myself. It is very interesting to see all the different methods you can do one task that I thought was so simple at first (thats my own ignorance ).


I know this is kinda a big project for me to do for my final year at university, thats why i have actually started the project 6 months before I even start my final year. So I have a year to get a very basic multiplayer game working. I would like to think that would be adequate time for this. I am only looking to have 3/4 players just be able to move around, no scene even, collision detection or etc. Just showing the movements of one player moving in real time on other machines.

Does anyone know where I can get specific information on how the object visit pattern works? I would like to learn more about it to get a more in-depth understanding of it.

Thanks once again for all the help everyone has commented on this thread.

agisler

The template is what allows you create different streams of any type.[/quote]

Sort of. The template allows you to use any type that simply implements the 'visit' function, since that is the function that is being called. The function must return a reference to itself. In a different context, you could write a class that is not a 'stream' oriented class and still be able to use the same syntax since that is how templates work.

The class therefore tells you how to read in the different types of streams.[/quote]

Correct, the implementation of the visit function determines what is done with the data.

Also the buf.push_back((i >> 24) & oxff); code this tells you where to start reading the data from.[/quote]

Not quite, no. In that code, network endian (big) order is being used to write out the integer. This is done so different platforms can share data and each can interpret it correctly. For more information check out endianness.

I am a bit uncertain about something as well. Is each InStream function reading a whole structure based on the type that comes in or just reading the individual data types that come in? Does that make sense?[/quote]

It reads each individual field as implemented by the visit function. Larger complex types are read by processing its simpler types. So in the end, you visit the root object, then all of its objects marked for visitation get visited, and this continues until the entire object has been read.

For example here's some pseduo-code:

// main scope
SeralizeStream ss;
Human human;
visist( human, ss ); // Star the serialization chain

...

void Hand(Hand & hand, Stream & stream )
{
stream.visit( "index", hand.index_finger ); // serialize this specific type to the stream, the end of the chain for this branch
...
}

void visit( Arms & arms, Stream & stream )
{
...
}

void visit( Legs & legs, Stream & stream )
{
...
}

void visit( Human & human, Stream & stream )
{
visit( human.arms, stream ); // Start the serialization chain down this type now
visit( human.legs stream ); // Start the serialization chain down this type now
}



The key thing to be aware of is "global visit" and "stream visit". Global visit starts a de/serilization chain of a more complex type while stream visist de/seralizes primitive data to the stream itself. So you would call global visit on user types, but never primitive objects. Likewise, you can only write primitive data types to a stream (binary representation), so that's why you don't call stream visit on more complex types.

Thanks for the additional comments. The benefits are more clear now as well. I do understand for the most part the object visitor code you have shown me (not the real world example). From what I understand it works very similar to hplus code.[/quote]
Yeap, I just took hplus code and added more stuff to it. It's great to have a wonderful mod like he is around the forums. For another example, I posted a reply in this thread that relates.

I know this is kinda a big project for me to do for my final year at university, thats why i have actually started the project 6 months before I even start my final year. So I have a year to get a very basic multiplayer game working. I would like to think that would be adequate time for this. I am only looking to have 3/4 players just be able to move around, no scene even, collision detection or etc. Just showing the movements of one player moving in real time on other machines. [/quote]

Sounds good. A simple tech demo of networked interactions is something that is pretty simple and good to start with. However, you need to make a chat program first as that is more suitable for your initial task. Writing a non network game can be challenging enough if you are not a game programmer and have a lot of experience. Trying to write a networked game when you don't have network experience either is all the more harder!

Starting with a chat application (something like IRC setup more so than a p2p messenger) would help you focus on the networking first. You want to allow users to create channels, change their names, and ensure messages are able to be sent to all intended recipients. Once you get comfortable with the process, you can apply those concepts to your simple game

The reason you don't want to just go right into player movement in a world is because the concept of player movement in a game can be a lot more complex than desirable depending on what type of game it is. For the sake of just getting familiar with this stuff, implementing the movement in an unrealistic way won't help you much because it's simple message passing. That same concept of message passing is why a chat application is more suitable to start with.

Does anyone know where I can get specific information on how the object visit pattern works? I would like to learn more about it to get a more in-depth understanding of it. [/quote]

You won't really need any more information about the design itself; you have everything you need. However, it sounds like you need to learn quite a bit more C++ to really understand what is going on. What compiler and IDE (if any) are you working with now?

To summarize the design:

1. Write a "stream reader" and "stream writer" class. (Mine were called DeseralizeStream and SeralizeStream. hplus's were called InStream and presumably OutStream). These classes must implement 'visit' functions for each data type you wish to serialize to the stream. The concept of serializing data needs to take into account the endianness, so that is why there is so much code to support both little and big endian (it's effectively 2x the code). Most people just choose one format and stick with it, but being able to support both is a bonus.

2. Write free standing templated visit functions for each higher level object type you will be serializing. The function signature should be: "template< typename Stream >Stream & visit( YOURTYPE & , Stream & )". Now, inside this function, you simply call the parameter stream's visit function on all member variables that should be serialized. You choose which ones you need. The nice thing about this design (from some perspectives) is you get control over what gets serialized and what does not. Only the member variables that you explicitly call visit on will get visited!

At this point, you will now be able to serialize and deserialize objects into a blob of data. However, all you have is a "payload". You have to attach more accompanying data before sending it across the network so it's meaningful.

3. Implement your own network protocol (assuming you already have a network library/framework to use). This will allow you to take the payload of your messages (what the visit pattern gives you as output) and attach what type of data it is (usually designated by an 'opcode') as well as the size. Remember TCP is a stream, so you must send the size of the data that is following. Once you have your own protocol, you can now just communicate between endpoints and worry about the actual network logic. If you are using UDP, then everything stays the same, except your protocol is slightly different. However, that is a different topic, so in context of this thread, everything is the same in regards to the visit pattern.

That's literally all there is to it.

An analogy of how it works would be, consider using a fork and a spoon for a meal. That is the traditional method of having WriteType and ReadType functions in different sections of logic. The visitor pattern would be like using a spork. You still have your spoon and fork, it's just in one tool (thanks to how C++ works, via templates and overloading). In the end, the functionality is the same, data still has to get written and read, you just go about different ways to implement it (and each have their pros and cons).

If you were using a language that did not support function overloading or templates, then this type of design could not be as efficiently expressed in the language. You would end up with similar code, just a lot more of it (minus the benefits already discussed).

Anyways, hope that helps. Good luck with your learning!