# Packet data types

This topic is 2402 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

Hello,

I have been reading all over about network game programming and they all seem to be somewhat concerned about being able to re-send a packet if it was lost. I don't understand why you would want to be able to resend a packet when your running a real time game. Am I missing something here?

My more important question is regarding sending out different data types. At the moment the network part of my game is insignificant. It only sends out a struct on the loopback address containing data such as the sequence, ack and position information of my one 3d model. What I want to know is what if I want to send out a stuct that contains slightly different information and data types? At the moment my game simply receives data into an identical struct. I can't imagine its efficient to have a struct that contains every bit of data that you may ever need. I hope this is not an obvious answer but i have tried to figure this out and look it up with no luck.

agisler

##### Share on other sites
You've got the right idea (except that you want to use some proper serialization instead of a struct--I like POSH).

However, you're going to eventually have data that HAS to get there and be in the proper order (login authorization, chat messages, trade, etc). You don't want to be resending huge chat messages over and over until you get an ack.

Just let enet handle this for you, it's the cleanest simplest UDP libarary I've seen.

##### Share on other sites
Thanks for the help,
I can't believe I got the right idea. Seems too simple

I have read the info regarding posh on there website, however its a bit over my head. Could you please explain to me what it does in a quick dumb downed version From what I gather its basically a custom data type? would this be an accurate statement?

This September I will be in my final year of a university studying computer networks. The networking part of the game I am building is what my final year project will be. I want to write my own code for this reason. It will not have any chatting or etc.. All I want to accomplish is a very basic flow control with some reliability in the areas I need.However I do plan on using ENET for my backup if all else fails

Thanks for the help.

agisler

##### Share on other sites

I have read the info regarding posh on there website, however its a bit over my head.

I've never used that library, but all it seems to do is to convert between big- and little-endian elements in memory, plus define some pre-defined constants based on what computer you're building it for. You can do almost all of this with the simple functions htons() and htonl() that are built into the sockets library. (except for the 64-bit versions)

In general, serialization means that you turn a high-level data structure into a low-level stream of bytes (kind of like saving to a file), and then back again. The stream of bytes should be self-contained given the known state of the serializer, but, for games, doesn't generally need to be self-explanatory. Thus, XML (where you name each field explicitly) is probably too verbose -- in a game, you know that when you see a "player position" packet, you will see (perhaps) a short for the player id, followed by three floats for player X, Y and Z.

Finally, you should separate "messages" from "packets." You send messages between endpoints in your system. The underlying network uses packets to transfer data from one node to another. Generally, you want to put many messages into a single packet, so as to minimize the overhead of packet headers. Many games will schedule a packet to be sent 1, 5, 10, or 30 times per second, and will have a queue of messages, where the queue will be put into a packet when the time comes, and all those messages sent at once.
To make this un-packable on the other side, you typically want each message to be encoded with a message type, and perhaps a length field, so that it's easy to tell messages apart and dispatch each of them separately.

##### Share on other sites
Typically, what I've done for exchanging data, is have a typical Packet header structure, which contains data values that every packet would need, and i include payload for packet type specific data.

For example, in my MUNE networking engine (http://mune.will.jennings.name) , I had a packet header structure like this:

[source]

typedef struct
{
U16 uwType;
U16 uwToUsers;
U16 uwFromUser;
U16 uwExtra;
U32 ulSize;
char pcData[1];
} tMunePacket;
[/source]

So, every message would be parsed based on it's type. If the message type required more data, it would be stored in the pcData. For example, if the message type was PLAYER_STATUS_UPDATE, the data in pcData may be a structure which has position and player status info. you could read and write it like this:

[source]

typedef struct
{
U32 u32PlayerId;
float fXpos;
float fYpos;
U32 u32PlayerHealth;

} tPlayerStatus;

tPlayerStatus gMainPlayerStatus;

.
.
.
tMunePacket *pPacket;

/* allocate room for header and data */
pPacket = (tMunePacket *)malloc(sizeof(tMunePacket ) + sizeof (tPlayerStatus));

/* fill in pPacket header data */
pPacket->uwType = PLAYER_STATUS_UPDATE;
/* copy player data in payload */
memcpy(pPacket->pcData, &gMainPlayerStatus, sizeof(tPlayerStatus));
[/source]

to read it out, you'd do this:
[source]

tPlayerStatus CurrentPlayerStatus;

switch (pPacket->u16MsgType)
{
case PLAYER_STATUS_UPDATE:
/* read player data out */
memcpy(&CurrentPlayerStatus, pPacket->pcData, sizeof(tPlayerStatus );
break;
}

[/source]

Just an example of how I've done it in the past.

##### Share on other sites
I really think you're better off defining packets as functions as opposed to types. (Well, functions are better than types in general so )

1. Endianness
3. Flexibility

Store the data wherever it is used by the game code, then write it to the packet in a function.

 void PlayerStatus(Packet& packet, const Player& player) { packet.WriteU16(player.id); // etc }

Your Packet class would store an internal byte array, and provide methods to read/write exact data types. RakNet has a decent implementation.

##### Share on other sites

 void PlayerStatus(Packet& packet, const Player& player) { packet.WriteU16(player.id); // etc }

Actually, if you want to go that way, you probably want to declare packets (or serialization) as visitors.

 struct MyStruct { int someId; std::string someString; float someValue; }; template<typename Stream> Stream &visit(MyStruct &p, Stream &strm) { return strm.visit("someId", p.someId) .visit("someString", p.someString) .visit("someValue", p.someValue); } 

Now, you can have one stream class that reads a struct and writes a byte stream, and another stream class that reads a byte stream and visits a struct.

 class InStream { public: std::vector<unsigned char> buf; InStream &visit(char const *name, int const &i) { buf.push_back((i >> 24) & 0xff); buf.push_back((i >> 16) & 0xff); buf.push_back((i >> 8) & 0xff); buf.push_back(i & 0xff); return *this; } InStream &visit(char const *name, std::string const &str) { if (str.size() > 255) throw std::exception("too long string"); buf.push_back(str.size()); buf.insert(buf.end(), str.begin(), str.end()); return *this; } ... 

The reason you pass the name of the field in as well as the field reference is that you may want to also marshal to JSON, XML, or an editor UI of some sort, which will need the name.

##### Share on other sites
Ok you guys are blowing my head away lol I thought this was a simple question that would have a simple answer I was wrong.

I like your suggestion BeerNutts, mostly cause its the one I can understand Just checking if I understand you correctly, all your basically doing is copying the pcData into a char? Of which the char is inside a struct that defines what type of packet it is. Is that about right?

As for the serialization this is quite difficult for me to understand at this point. Because of my lack of understanding I am struggling with the code and the benefits of using this method.

[color="#1C2837"]
Actually, if you want to go that way, you probably want to declare packets (or serialization) as visitors.

[color="#000088"]struct [color="#660066"]MyStruct [color="#666600"]{ [color="#000088"]int[color="#000000"] someId[color="#666600"];[color="#000000"]
std[color="#666600"]::[color="#000088"]string[color="#000000"] someString[color="#666600"]; [color="#000088"]float[color="#000000"] someValue[color="#666600"]; [color="#666600"]}; [color="#000088"]template[color="#666600"]<[color="#000088"]typename [color="#660066"]Stream[color="#666600"]> [color="#660066"]Stream [color="#666600"]&[color="#000000"]visit[color="#666600"]([color="#660066"]MyStruct [color="#666600"]&[color="#000000"]p[color="#666600"], [color="#660066"]Stream [color="#666600"]&[color="#000000"]strm[color="#666600"]) [color="#666600"]{ [color="#000088"]return[color="#000000"] strm[color="#666600"].[color="#000000"]visit[color="#666600"]([color="#008800"]"someId"[color="#666600"],[color="#000000"] p[color="#666600"].[color="#000000"]someId[color="#666600"]) [color="#666600"].[color="#000000"]visit[color="#666600"]([color="#008800"]"someString"[color="#666600"],[color="#000000"] p[color="#666600"].[color="#000000"]someString[color="#666600"]) [color="#666600"].[color="#000000"]visit[color="#666600"]([color="#008800"]"someValue"[color="#666600"],[color="#000000"] p[color="#666600"].[color="#000000"]someValue[color="#666600"]); [color="#666600"]}

Now, you can have one stream class that reads a struct and writes a byte stream, and another stream class that reads a byte stream and visits a struct.

[color="#000088"]class [color="#660066"]InStream [color="#666600"]{ [color="#000088"]public[color="#666600"]:[color="#000000"]
std[color="#666600"]::[color="#000000"]vector[color="#666600"]<[color="#000088"]unsigned [color="#000088"]char[color="#666600"]>[color="#000000"] buf[color="#666600"]; [color="#660066"]InStream [color="#666600"]&[color="#000000"]visit[color="#666600"]([color="#000088"]char [color="#000088"]const [color="#666600"]*[color="#000000"]name[color="#666600"], [color="#000088"]int [color="#000088"]const [color="#666600"]&[color="#000000"]i[color="#666600"]) [color="#666600"]{[color="#000000"]
buf[color="#666600"].[color="#000000"]push_back[color="#666600"](([color="#000000"]i [color="#666600"]>> [color="#006666"]24[color="#666600"]) [color="#666600"]& [color="#006666"]0xff[color="#666600"]);[color="#000000"]
buf[color="#666600"].[color="#000000"]push_back[color="#666600"](([color="#000000"]i [color="#666600"]>> [color="#006666"]16[color="#666600"]) [color="#666600"]& [color="#006666"]0xff[color="#666600"]);[color="#000000"]
buf[color="#666600"].[color="#000000"]push_back[color="#666600"](([color="#000000"]i [color="#666600"]>> [color="#006666"]8[color="#666600"]) [color="#666600"]& [color="#006666"]0xff[color="#666600"]);[color="#000000"]
buf[color="#666600"].[color="#000000"]push_back[color="#666600"]([color="#000000"]i [color="#666600"]& [color="#006666"]0xff[color="#666600"]); [color="#000088"]return [color="#666600"]*[color="#000088"]this[color="#666600"]; [color="#666600"]} [color="#660066"]InStream [color="#666600"]&[color="#000000"]visit[color="#666600"]([color="#000088"]char [color="#000088"]const [color="#666600"]*[color="#000000"]name[color="#666600"],[color="#000000"] std[color="#666600"]::[color="#000088"]string [color="#000088"]const [color="#666600"]&[color="#000000"]str[color="#666600"]) [color="#666600"]{ [color="#000088"]if [color="#666600"]([color="#000000"]str[color="#666600"].[color="#000000"]size[color="#666600"]() [color="#666600"]> [color="#006666"]255[color="#666600"]) [color="#000088"]throw[color="#000000"] std[color="#666600"]::[color="#000000"]exception[color="#666600"]([color="#008800"]"too long string"[color="#666600"]);[color="#000000"]
buf[color="#666600"].[color="#000000"]push_back[color="#666600"]([color="#000000"]str[color="#666600"].[color="#000000"]size[color="#666600"]());[color="#000000"]
buf[color="#666600"].[color="#000000"]insert[color="#666600"]([color="#000000"]buf[color="#666600"].[color="#000088"]end[color="#666600"](),[color="#000000"] str[color="#666600"].[color="#000088"]begin[color="#666600"](),[color="#000000"] str[color="#666600"].[color="#000088"]end[color="#666600"]()); [color="#000088"]return [color="#666600"]*[color="#000088"]this[color="#666600"]; [color="#666600"]} [color="#666600"]...[/quote]

From what I can understand your saying I have a class that reads a structure when the data comes in and then outputs the data to a file essentially? Then you have a class that does the reverse of that?

This is a bit over my head but I do not have a problem reading about it and trying to learn this. However what's the benefits of me doing it this way?

Once again thank you all for the help

agisler

##### Share on other sites
[font=arial, verdana, tahoma, sans-serif][size=2]

Ok you guys are blowing my head away lol I thought this was a simple question that would have a simple answer I was wrong.

I like your suggestion BeerNutts, mostly cause its the one I can understand Just checking if I understand you correctly, all your basically doing is copying the pcData into a char? Of which the char is inside a struct that defines what type of packet it is. Is that about right?

Well, it's kind of a trick. The structure's last item is char pcData[1];

Technically, this defines an array of chars of length 1; however, when I allocate the memory for the structure, I have size for the structure, plus size for the "payload" data. What this actually does is points pcData to allocated memory the size of the payload structure. It allows you to pass the packet structure with the payload data on the end, which is how I handle serialization in C. So, I can memcpy any structure into pcData, send the full packet structure , plus the payload length, to the server. When the client gets the data, he reads the packet header, determines the type, and can copy the payload data from pcData into whatever structure type he wants.

In my example above, I was copying the payload data in the tPlayerStatus structure, which defines a certain players status.

This requires some working knowledge of pointers and memory allocation. That's why, in my MUNE engine, I only provide a function to define packets and append data for you:
 tMunePacket * MuneCreatePacket (U16 uwType, U16 uwTo, U16 uwFrom, U8 *pucData, U32 ulSize) 

(you can get the source code from mune.will.jennings.name as well if you want)

Realize, all my code is in C. The other guys are offering options in C++ if you want to go that route.
[/font]

##### Share on other sites

As for the serialization this is quite difficult for me to understand at this point. Because of my lack of understanding I am struggling with the code and the benefits of using this method.

...

From what I can understand your saying I have a class that reads a structure when the data comes in and then outputs the data to a file essentially? Then you have a class that does the reverse of that?

You may be better off doing more programming on a single machine, until you are more at ease with memory, bytes, how data structures are represented, and other such systems programming details.

Anyway, yes, a TCP stream of packets, and a file on disk, are actually very much similar. What my proposed marshaling does is define a particular struct per message type, and then build *one* template function per struct. All this function does is visit each struct member in order, calling into some template argument for each. You can then provide two classes: an "input reader" and an "output writer," which lets you read into the struct, or write out of the struct, depending on which you pass in.

What's extra elegant about this is that you can also write a class that does things like "build a GUI" or "send to log file" or whatever -- when you have the struct, and the visitor function, you can do many things to any instance of that struct, by simply building a different "stream" class, and using the same visitor. This allows you to separate the concerns of what the data is (the struct), how to describe what the data is (the visitor function), and what you do with a description of the data (the different class implementations).

This means you don't have to write separate "send()" and "receive()" and "editInGUI()" and "dumpAsXML()" and "logToFile()" functions for the same data struct, which is a pretty big win once your program becomes bigger.

##### Share on other sites

You may be better off doing more programming on a single machine, until you are more at ease with memory, bytes, how data structures are represented, and other such systems programming details.

Anyway, yes, a TCP stream of packets, and a file on disk, are actually very much similar. What my proposed marshaling does is define a particular struct per message type, and then build *one* template function per struct. All this function does is visit each struct member in order, calling into some template argument for each. You can then provide two classes: an "input reader" and an "output writer," which lets you read into the struct, or write out of the struct, depending on which you pass in.

What's extra elegant about this is that you can also write a class that does things like "build a GUI" or "send to log file" or whatever -- when you have the struct, and the visitor function, you can do many things to any instance of that struct, by simply building a different "stream" class, and using the same visitor. This allows you to separate the concerns of what the data is (the struct), how to describe what the data is (the visitor function), and what you do with a description of the data (the different class implementations).

This means you don't have to write separate "send()" and "receive()" and "editInGUI()" and "dumpAsXML()" and "logToFile()" functions for the same data struct, which is a pretty big win once your program becomes bigger.

@agisler
The C++ Middleware Writer -- http://webEbenezer.n...ntegration.html -- is an on line code generator that writes C++ marshalling code based on high-level user input. It automates the creation of functions that hplus has mentioned.

Brian Wood
Ebenezer Enterprises
http://webEbenezer.net

##### Share on other sites

This is a bit over my head but I do not have a problem reading about it and trying to learn this. However what's the benefits of me doing it this way?

To put simply, rather than having to write 4 pieces of logic: stream writer, stream reader, object writer, object reader, you will only have to write 3: stream writer, stream reader, object visitor. It might not seem like much at first, but as your project gets larger, the difference between the two methods' amount of code is huge. In addition, when you use the object writer and object reader approach, you have two pieces of logic you have to maintain and keep track up for maintenance whereas the object visitor is only one piece of logic.

By going the object visitor route, you further decrease development time when you have base types that can be reused. For example, let's say you have 5 different messages that all contain the same sequence of data, like entity id, X, Y, Z. In the object reader/writer approach, you will simply have logic to read/write each of those fields individually in all of your functions. In the object visitor approach, if you combined those 4 fields into a base type, you would only have to write the visit function logic once for that type, then reap the benefits of being able to reuse it for any message that uses it. So rather than having 8 lines of actual reading/writing code total, you only have 1. That is because the visitor pattern handles both reading and writing!

Here is a real world example. Consider the following structure that contains data about a security protocol:
[spoiler]
 struct Data_Security { unsigned char mode; unsigned __int64 initial_key; unsigned int seed_count; unsigned int crc_seed; unsigned __int64 handshake_key; unsigned int g; unsigned int p; unsigned int A; Data_Security() { mode = 0; initial_key = 0; seed_count = 0; crc_seed = 0; handshake_key = 0; g = 0; p = 0; A = 0; } }; [/spoiler]

Using the object writer/reader approach, we would have the following two functions:
[spoiler] Data_Security FromStream( StreamReader stream ) { Data_Security object; object.mode = stream.ReadUInt8(); if( object.mode & 2 ) { object.initial_key = stream.ReadUInt64(); } if( object.mode & 4 ) { object.seed_count = stream.ReadUInt32(); object.crc_seed = stream.ReadUInt32(); } if( object.mode & 8 ) { object.handshake_key = stream.ReadUInt64(); object.g = stream.ReadUInt32(); object.p = stream.ReadUInt32(); object.A = stream.ReadUInt32(); } if( object.mode & 16 ) { object.handshake_key = stream.ReadUInt64(); } } void ToStream( StreamWriter stream, const Data_Security & object ) { stream.WriteUInt8( object.mode ); if( object.mode & 2 ) { stream.WriteUInt64( object.initial_key ); } if( object.mode & 4 ) { stream.WriteUInt32( object.seed_count ); stream.WriteUInt32( object.crc_seed ); } if( object.mode & 8 ) { stream.WriteUInt64( object.handshake_key ); stream.WriteUInt32( object.g ); stream.WriteUInt32( object.p ); stream.WriteUInt32( object.A ); } if( object.mode & 16 ) { stream.WriteUInt64( object.handshake_key ); } } [/spoiler]

Where the WriteXXX / ReadXXX functions are coded as part of the StreamWriter / StreamReader class.

Now, for the object visitor pattern, we only have one function:
[spoiler] template< typename Stream > Stream & visit( Data_Security & value, Stream & stream ) { stream.visit( "mode", value.mode ); if( value.mode & 2 ) { stream.visit( "initial_key", value.initial_key ); } if( value.mode & 4 ) { stream.visit( "seed_count", value.seed_count ); stream.visit( "crc_seed", value.crc_seed ); } if( value.mode & 8 ) { stream.visit( "handshake_key", value.handshake_key ); stream.visit( "g", value.g ); stream.visit( "p", value.p ); stream.visit( "A", value.A ); } if( value.mode & 16 ) { stream.visit( "handshake_key", value.handshake_key ); } return stream; } [/spoiler]

We still have both StreamReader and StreamWriter classes, but rather than them defining Read/Write named functions, they all use the same "visit" function with different logic depending on the object.

So we are taking advantage of the way C++ works to drastically cut down on the work and code needed to implement object serialization. The more types you have, the more visit functions you do have to write, but you only have to write them once, so you can easily reuse them in the future. As mentioned before, as your project grows, the object visitor pattern pays for itself.

The object reader/writer code shown is typically how you see people do it. I myself used that style for years because I was unaware of the visitor pattern. Now that I understand it better, I can see how beneficial it is and how there really is no reason to use the object reader/writer method because everything you can accomplish there, you can accomplish with the visitor pattern; you just might need to add a state object to know some extra information.

Here's some simple examples of more complete visitor stream classes shown in hplus's earlier post:
SeralizeStream
[spoiler] class SeralizeStream { private: std::vector< unsigned char > m_buffer; public: SeralizeStream & visit( const std::string & name, const std::string & value ) { #ifdef _DEBUG std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << std::endl; #endif size_t size = value.size(); do { if( size >= 255 ) { m_buffer.push_back( 255 ); size -= 254; } else { m_buffer.push_back( static_cast< unsigned char >( size ) ); size -= size; } } while( size != 0 ); m_buffer.insert( m_buffer.end(), value.begin(), value.end() ); return *this; } SeralizeStream & visit( const std::string & name, const unsigned char & value ) { #ifdef _DEBUG std::cout << "[" << __FUNCTION__<< "] " << name << " => " << (int)value << " (0x" << std::hex << (int)value << std::dec << ")" << std::endl; #endif m_buffer.push_back( value ); return *this; } SeralizeStream & visit( const std::string & name, const signed char & value ) { #ifdef _DEBUG std::cout << "[" << __FUNCTION__<< "] " << name << " => " << (int)value << " (0x" << std::hex << (int)value << std::dec << ")" << std::endl; #endif m_buffer.push_back( value ); return *this; } SeralizeStream & visit( const std::string & name, const unsigned short & value ) { #ifdef _DEBUG std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << " (0x" << std::hex << value << std::dec << ")" << std::endl; #endif #if ENDIAN_BIG == 1 m_buffer.push_back( ( value >> 8 ) & 0xFF ); m_buffer.push_back( ( value >> 0 ) & 0xFF ); #elif ENDIAN_LITTLE == 1 m_buffer.push_back( ( value >> 0 ) & 0xFF ); m_buffer.push_back( ( value >> 8 ) & 0xFF ); #endif return *this; } SeralizeStream & visit( const std::string & name, const signed short & value ) { #ifdef _DEBUG std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << " (0x" << std::hex << value << std::dec << ")" << std::endl; #endif #if ENDIAN_BIG == 1 m_buffer.push_back( ( value >> 8 ) & 0xFF ); m_buffer.push_back( ( value >> 0 ) & 0xFF ); #elif ENDIAN_LITTLE == 1 m_buffer.push_back( ( value >> 0 ) & 0xFF ); m_buffer.push_back( ( value >> 8 ) & 0xFF ); #endif return *this; } SeralizeStream & visit( const std::string & name, const unsigned int & value ) { #ifdef _DEBUG std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << " (0x" << std::hex << value << std::dec << ")" << std::endl; #endif #if ENDIAN_BIG == 1 m_buffer.push_back( ( value >> 24 ) & 0xFF ); m_buffer.push_back( ( value >> 16 ) & 0xFF ); m_buffer.push_back( ( value >> 8 ) & 0xFF ); m_buffer.push_back( ( value >> 0 ) & 0xFF ); #elif ENDIAN_LITTLE == 1 m_buffer.push_back( ( value >> 0 ) & 0xFF ); m_buffer.push_back( ( value >> 8 ) & 0xFF ); m_buffer.push_back( ( value >> 16 ) & 0xFF ); m_buffer.push_back( ( value >> 24 ) & 0xFF ); #endif return *this; } SeralizeStream & visit( const std::string & name, const signed int & value ) { #ifdef _DEBUG std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << " (0x" << std::hex << value << std::dec << ")" << std::endl; #endif #if ENDIAN_BIG == 1 m_buffer.push_back( ( value >> 24 ) & 0xFF ); m_buffer.push_back( ( value >> 16 ) & 0xFF ); m_buffer.push_back( ( value >> 8 ) & 0xFF ); m_buffer.push_back( ( value >> 0 ) & 0xFF ); #elif ENDIAN_LITTLE == 1 m_buffer.push_back( ( value >> 0 ) & 0xFF ); m_buffer.push_back( ( value >> 8 ) & 0xFF ); m_buffer.push_back( ( value >> 16 ) & 0xFF ); m_buffer.push_back( ( value >> 24 ) & 0xFF ); #endif return *this; } SeralizeStream & visit( const std::string & name, const unsigned __int64 & value ) { #ifdef _DEBUG std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << " (0x" << std::hex << value << std::dec << ")" << std::endl; #endif #if ENDIAN_BIG == 1 m_buffer.push_back( ( value >> 56 ) & 0xFF ); m_buffer.push_back( ( value >> 48 ) & 0xFF ); m_buffer.push_back( ( value >> 40 ) & 0xFF ); m_buffer.push_back( ( value >> 32 ) & 0xFF ); m_buffer.push_back( ( value >> 24 ) & 0xFF ); m_buffer.push_back( ( value >> 16 ) & 0xFF ); m_buffer.push_back( ( value >> 8 ) & 0xFF ); m_buffer.push_back( ( value >> 0 ) & 0xFF ); #elif ENDIAN_LITTLE == 1 m_buffer.push_back( ( value >> 0 ) & 0xFF ); m_buffer.push_back( ( value >> 8 ) & 0xFF ); m_buffer.push_back( ( value >> 16 ) & 0xFF ); m_buffer.push_back( ( value >> 24 ) & 0xFF ); m_buffer.push_back( ( value >> 32 ) & 0xFF ); m_buffer.push_back( ( value >> 40 ) & 0xFF ); m_buffer.push_back( ( value >> 48 ) & 0xFF ); m_buffer.push_back( ( value >> 56 ) & 0xFF ); #endif return *this; } SeralizeStream & visit( const std::string & name, const signed __int64 & value ) { #ifdef _DEBUG std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << " (0x" << std::hex << value << std::dec << ")" << std::endl; #endif #if ENDIAN_BIG == 1 m_buffer.push_back( ( value >> 56 ) & 0xFF ); m_buffer.push_back( ( value >> 48 ) & 0xFF ); m_buffer.push_back( ( value >> 40 ) & 0xFF ); m_buffer.push_back( ( value >> 32 ) & 0xFF ); m_buffer.push_back( ( value >> 24 ) & 0xFF ); m_buffer.push_back( ( value >> 16 ) & 0xFF ); m_buffer.push_back( ( value >> 8 ) & 0xFF ); m_buffer.push_back( ( value >> 0 ) & 0xFF ); #elif ENDIAN_LITTLE == 1 m_buffer.push_back( ( value >> 0 ) & 0xFF ); m_buffer.push_back( ( value >> 8 ) & 0xFF ); m_buffer.push_back( ( value >> 16 ) & 0xFF ); m_buffer.push_back( ( value >> 24 ) & 0xFF ); m_buffer.push_back( ( value >> 32 ) & 0xFF ); m_buffer.push_back( ( value >> 40 ) & 0xFF ); m_buffer.push_back( ( value >> 48 ) & 0xFF ); m_buffer.push_back( ( value >> 56 ) & 0xFF ); #endif return *this; } SeralizeStream & visit( const std::string & name, const float & value ) { #ifdef _DEBUG std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << std::endl; #endif return visit( name, *( unsigned int * )( &value ) ); } SeralizeStream & visit( const std::string & name, const double & value ) { #ifdef _DEBUG std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << std::endl; #endif return visit( name, *( unsigned __int64 * )( &value ) ); } std::vector< unsigned char > & Buffer() { return m_buffer; } void Clear() { m_buffer.clear(); } }; [/spoiler]

DeseralizeStream
[spoiler] class DeseralizeStream { private: std::vector<unsigned char> & m_buffer; size_t m_index; public: DeseralizeStream(std::vector<unsigned char> & buffer) : m_buffer( buffer ), m_index( 0 ) { } DeseralizeStream & visit( const std::string & name, std::string & value ) { size_t size = 0; while( m_buffer[ m_index ] == 255 ) { size += 254; ++m_index; } size += m_buffer[ m_index++ ]; value.resize( size ); std::copy( m_buffer.begin() + m_index, m_buffer.begin() + m_index + size, value.begin() ); m_index += size; #ifdef _DEBUG std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << std::endl; #endif return *this; } DeseralizeStream & visit( const std::string & name, unsigned char & value ) { value = (unsigned char)m_buffer[ m_index++ ]; #ifdef _DEBUG std::cout << "[" << __FUNCTION__<< "] " << name << " => " << (int)value << " (0x" << std::hex << (int)value << std::dec << ")" << std::endl; #endif return *this; } DeseralizeStream & visit( const std::string & name, signed char & value ) { value = (signed char)m_buffer[ m_index++ ]; #ifdef _DEBUG std::cout << "[" << __FUNCTION__<< "] " << name << " => " << (int)value << " (0x" << std::hex << (int)value << std::dec << ")" << std::endl; #endif return *this; } DeseralizeStream & visit( const std::string & name, unsigned short & value ) { #if ENDIAN_BIG == 1 value = (unsigned short)m_buffer[ m_index++ ] << 8 | (unsigned short)m_buffer[ m_index++ ] << 0; #elif ENDIAN_LITTLE == 1 value = (unsigned short)m_buffer[ m_index++ ] << 0 | (unsigned short)m_buffer[ m_index++ ] << 8; #endif #ifdef _DEBUG std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << " (0x" << std::hex << value << std::dec << ")" << std::endl; #endif return *this; } DeseralizeStream & visit( const std::string & name, signed short & value ) { #if ENDIAN_BIG == 1 value = (signed short)m_buffer[ m_index++ ] << 8 | (signed short)m_buffer[ m_index++ ] << 0; #elif ENDIAN_LITTLE == 1 value = (signed short)m_buffer[ m_index++ ] << 0 | (signed short)m_buffer[ m_index++ ] << 8; #endif #ifdef _DEBUG std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << " (0x" << std::hex << value << std::dec << ")" << std::endl; #endif return *this; } DeseralizeStream & visit( const std::string & name, unsigned int & value ) { #if ENDIAN_BIG == 1 value = (unsigned int)m_buffer[ m_index++ ] << 24 | (unsigned int)m_buffer[ m_index++ ] << 16 | (unsigned int)m_buffer[ m_index++ ] << 8 | (unsigned int)m_buffer[ m_index++ ] << 0; #elif ENDIAN_LITTLE == 1 value = (unsigned int)m_buffer[ m_index++ ] << 0 | (unsigned int)m_buffer[ m_index++ ] << 8 | (unsigned int)m_buffer[ m_index++ ] << 16 | (unsigned int)m_buffer[ m_index++ ] << 24; #endif #ifdef _DEBUG std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << " (0x" << std::hex << value << std::dec << ")" << std::endl; #endif return *this; } DeseralizeStream & visit( const std::string & name, signed int & value ) { #if ENDIAN_BIG == 1 value = (signed int)m_buffer[ m_index++ ] << 24 | (signed int)m_buffer[ m_index++ ] << 16 | (signed int)m_buffer[ m_index++ ] << 8 | (signed int)m_buffer[ m_index++ ] << 0; #elif ENDIAN_LITTLE == 1 value = (signed int)m_buffer[ m_index++ ] << 0 | (signed int)m_buffer[ m_index++ ] << 8 | (signed int)m_buffer[ m_index++ ] << 16 | (signed int)m_buffer[ m_index++ ] << 24; #endif #ifdef _DEBUG std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << " (0x" << std::hex << value << std::dec << ")" << std::endl; #endif return *this; } DeseralizeStream & visit( const std::string & name, unsigned __int64 & value ) { #if ENDIAN_BIG == 1 value = (unsigned __int64)m_buffer[ m_index++ ] << 56 | (unsigned __int64)m_buffer[ m_index++ ] << 48 | (unsigned __int64)m_buffer[ m_index++ ] << 40 | (unsigned __int64)m_buffer[ m_index++ ] << 32 | (unsigned __int64)m_buffer[ m_index++ ] << 24 | (unsigned __int64)m_buffer[ m_index++ ] << 16 | (unsigned __int64)m_buffer[ m_index++ ] << 8 | (unsigned __int64)m_buffer[ m_index++ ] << 0; #elif ENDIAN_LITTLE == 1 value = (unsigned __int64)m_buffer[ m_index++ ] << 0 | (unsigned __int64)m_buffer[ m_index++ ] << 8 | (unsigned __int64)m_buffer[ m_index++ ] << 16 | (unsigned __int64)m_buffer[ m_index++ ] << 24 | (unsigned __int64)m_buffer[ m_index++ ] << 32 | (unsigned __int64)m_buffer[ m_index++ ] << 40 | (unsigned __int64)m_buffer[ m_index++ ] << 48 | (unsigned __int64)m_buffer[ m_index++ ] << 56; #endif #ifdef _DEBUG std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << " (0x" << std::hex << value << std::dec << ")" << std::endl; #endif return *this; } DeseralizeStream & visit( const std::string & name, signed __int64 & value ) { #if ENDIAN_BIG == 1 value = (signed __int64)m_buffer[ m_index++ ] << 56 | (signed __int64)m_buffer[ m_index++ ] << 48 | (signed __int64)m_buffer[ m_index++ ] << 40 | (signed __int64)m_buffer[ m_index++ ] << 32 | (signed __int64)m_buffer[ m_index++ ] << 24 | (signed __int64)m_buffer[ m_index++ ] << 16 | (signed __int64)m_buffer[ m_index++ ] << 8 | (signed __int64)m_buffer[ m_index++ ] << 0; #elif ENDIAN_LITTLE == 1 value = (signed __int64)m_buffer[ m_index++ ] << 0 | (signed __int64)m_buffer[ m_index++ ] << 8 | (signed __int64)m_buffer[ m_index++ ] << 16 | (signed __int64)m_buffer[ m_index++ ] << 24 | (signed __int64)m_buffer[ m_index++ ] << 32 | (signed __int64)m_buffer[ m_index++ ] << 40 | (signed __int64)m_buffer[ m_index++ ] << 48 | (signed __int64)m_buffer[ m_index++ ] << 56; #endif #ifdef _DEBUG std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << " (0x" << std::hex << value << std::dec << ")" << std::endl; #endif return *this; } DeseralizeStream & visit( const std::string & name, float & value ) { unsigned int v; visit( name, v ); memcpy( &value, &v, 4 ); #ifdef _DEBUG std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << std::endl; #endif return *this; } DeseralizeStream & visit( const std::string & name, double & value ) { unsigned __int64 v; visit( name, v ); memcpy( &value, &v, 8 ); #ifdef _DEBUG std::cout << "[" << __FUNCTION__<< "] " << name << " => " << value << std::endl; #endif return *this; } }; [/spoiler]

They are pretty basic classes. More functions for vectors, lists, maps, etc... could be added as needed. Also the way you work with strings might vary. Some protocols use fixed size strings, some use a variable length variable size type similar to the one shown, and others just use a variable length fixed size type (as shown in hplus's post with a 1 byte length limitation). I think the float/double logic is correct, but I might be wrong. Another thing to be careful of is ensuring you are using portable types (I'm not purposefully for the sake of a simple test). There are a few gotchas here you have to be careful of if you are going cross-platform or 32/64-bit different architectures. The most annoying one is the differences between wchar_t size on gcc on linux (4 bytes usually) and the size on windows (2 bytes usually). If you tried to send a string from one platform to the other without keeping this in mind, you can be in for some real headaches!(I.e. Windows Client <-> Linux Proxy Server <-> Windows Server).

Anyways, hopefully that adds to the useful information in this thread. Also as a disclaimer, all code was written during the course of reading this thread, so it might have bugs, do not use it without understanding what it does first. Good luck!

##### Share on other sites

So we are taking advantage of the way C++ works to drastically cut down on the work and code needed to implement object serialization. The more types you have, the more visit functions you do have to write, but you only have to write them once, so you can easily reuse them in the future. As mentioned before, as your project grows, the object visitor pattern pays for itself.

Don't forget that you have to revisit the visit functions when you make changes to your types. With the C++ Middleware Writer you *don't* have to write or maintain visit functions.

Brian Wood
Ebenezer Enterprises
http://webEbenezer.net

##### Share on other sites
But you have to specify it somewhere, right? C++ lacks reflection. Plus if I remember correctly with your library you need to have code compiled by your server or something? That sounds far more complex to me than maintaining the visit functions.

##### Share on other sites

But you have to specify it somewhere, right?

No. The CMW (C++ Middleware Writer) rereads updated header files and recreates the marshalling code as needed.

C++ lacks reflection. Plus if I remember correctly with your library you need to have code compiled by your server or something? That sounds far more complex to me than maintaining the visit functions.
[/quote]

You have to have to submit some code to the CMW, but I don't think it is complex. Getting set up to use the CMW takes about 15 minutes. That includes downloading the prerequisite Loki library and setting up an account. I use the following architecture:

CMW (server)
|
|
direct program (runs once and exits)

Brian

There are two phases to setting up the CMW. I've described the first phase -- getting output from it -- the same output that comes in the archive. The second phase is beginning to use it in your project. The 15 minutes I mention is for getting the first phase working. The second phase though isn't difficult either.

##### Share on other sites

You have to have to submit some code to the CMW, but I don't think it is complex.

I still don't get why you're trying to do this as a server. No sane developer would actually place the health of his project in the hands of a server operated by some random guy.
Even when "some random guy" is really big (say, IBM size) you can get screwed, because whoever operates the server may at some point:
1) get hit by a bus
2) get hacked by some bad guys
3) decide that it's not profitable or fun
and you, as a developer, are cut off -- no more server available, and you can no longer build your project.

When you buy tools (rather than services,) you can keep running the tools for as long as you want.
Compare Blade3D -- a game development engine that was using a monthly payment/service model. At some point, they just decided to stop providing this service. Oops!

##### Share on other sites

I still don't get why you're trying to do this as a server. No sane developer would actually place the health of his project in the hands of a server operated by some random guy.
Even when "some random guy" is really big (say, IBM size) you can get screwed, because whoever operates the server may at some point:
1) get hit by a bus

Couldn't you throw in a "G-d forbid" here or there?

2) get hacked by some bad guys
3) decide that it's not profitable or fun
and you, as a developer, are cut off -- no more server available, and you can no longer build your project.

When you buy tools (rather than services,) you can keep running the tools for as long as you want.
Compare Blade3D -- a game development engine that was using a monthly payment/service model. At some point, they just decided to stop providing this service. Oops!
[/quote]

I'm sure you've heard of all the countries around the world that have over 20% unemployment. The official rate in the US is no longer really accurate -- http://www.investors...89-BLS-Rate.htm . Egypt, Spain, Greece -- it is a long list -- have very high unemployment. I provide the hosting for people around the world who can't afford to buy the service as a tool. I'm not really interested in marketing this to IBM and other well-known companies out there. I'm aiming for the little guy. There's no shortage of people who need help with their projects. Thanks to some lame excuses for "leaders", the world is in bad shape. Some people may not like my terms, but that doesn't change their situations. They like to eat and drink, have heat, buy clothes, buy their children toys... Anyway, I think its obvious that as I get more users my position becomes stronger -- I get better at preventing thieves (hacking), running the business and dodging buses.

The world is changing. I've long thought for example, that Microsoft is at a disadvantage to Google. Google wasn't afraid of the cloud; they embraced it. Microsoft goes around trying to get countries to crack down on software theft. I think that's a tough proposition and they are huge. I don't have to worry about things like that with this model.

Brian

##### Share on other sites
WOW! I have not checked the thread in the last few days as I have been doing my homework on this subject

I think now I understand the basic concepts of hplus code. So I will just check with you that I have the right idea of the code.

The template is what allows you create different streams of any type. The class therefore tells you how to read in the different types of streams. Also the  buf.push_back((i >> 24) & oxff);  code this tells you where to start reading the data from.

A few questions regarding it as well.

what library is the std::vector contained in?

I am a bit uncertain about something as well. Is each InStream function reading a whole structure based on the type that comes in or just reading the individual data types that come in? Does that make sense?

@Drew_Benton

Thanks for the additional comments. The benefits are more clear now as well. I do understand for the most part the object visitor code you have shown me (not the real world example). From what I understand it works very similar to hplus code.

@wood_Brian

Thanks for the input. However as this is going to be a final year university project, so I need to keep everything local so that I can work on it myself. It is very interesting to see all the different methods you can do one task that I thought was so simple at first (thats my own ignorance ).

I know this is kinda a big project for me to do for my final year at university, thats why i have actually started the project 6 months before I even start my final year. So I have a year to get a very basic multiplayer game working. I would like to think that would be adequate time for this. I am only looking to have 3/4 players just be able to move around, no scene even, collision detection or etc. Just showing the movements of one player moving in real time on other machines.

Does anyone know where I can get specific information on how the object visit pattern works? I would like to learn more about it to get a more in-depth understanding of it.

Thanks once again for all the help everyone has commented on this thread.

agisler

##### Share on other sites

what library is the std::vector contained in?

std::vector lives in the standard C++ library, in the header file "vector". (generally included with brackets)

##### Share on other sites
The template is what allows you create different streams of any type.[/quote]

Sort of. The template allows you to use any type that simply implements the 'visit' function, since that is the function that is being called. The function must return a reference to itself. In a different context, you could write a class that is not a 'stream' oriented class and still be able to use the same syntax since that is how templates work.

The class therefore tells you how to read in the different types of streams.[/quote]

Correct, the implementation of the visit function determines what is done with the data.

Also the  buf.push_back((i >> 24) & oxff);  code this tells you where to start reading the data from.[/quote]

Not quite, no. In that code, network endian (big) order is being used to write out the integer. This is done so different platforms can share data and each can interpret it correctly. For more information check out endianness.

I am a bit uncertain about something as well. Is each InStream function reading a whole structure based on the type that comes in or just reading the individual data types that come in? Does that make sense?[/quote]

It reads each individual field as implemented by the visit function. Larger complex types are read by processing its simpler types. So in the end, you visit the root object, then all of its objects marked for visitation get visited, and this continues until the entire object has been read.

For example here's some pseduo-code:
 // main scope SeralizeStream ss; Human human; visist( human, ss ); // Star the serialization chain ... void Hand(Hand & hand, Stream & stream ) { stream.visit( "index", hand.index_finger ); // serialize this specific type to the stream, the end of the chain for this branch ... } void visit( Arms & arms, Stream & stream ) { ... } void visit( Legs & legs, Stream & stream ) { ... } void visit( Human & human, Stream & stream ) { visit( human.arms, stream ); // Start the serialization chain down this type now visit( human.legs stream ); // Start the serialization chain down this type now } 

The key thing to be aware of is "global visit" and "stream visit". Global visit starts a de/serilization chain of a more complex type while stream visist de/seralizes primitive data to the stream itself. So you would call global visit on user types, but never primitive objects. Likewise, you can only write primitive data types to a stream (binary representation), so that's why you don't call stream visit on more complex types.

Thanks for the additional comments. The benefits are more clear now as well. I do understand for the most part the object visitor code you have shown me (not the real world example). From what I understand it works very similar to hplus code.[/quote]
Yeap, I just took hplus code and added more stuff to it. It's great to have a wonderful mod like he is around the forums. For another example, I posted a reply in this thread that relates.

I know this is kinda a big project for me to do for my final year at university, thats why i have actually started the project 6 months before I even start my final year. So I have a year to get a very basic multiplayer game working. I would like to think that would be adequate time for this. I am only looking to have 3/4 players just be able to move around, no scene even, collision detection or etc. Just showing the movements of one player moving in real time on other machines. [/quote]

Sounds good. A simple tech demo of networked interactions is something that is pretty simple and good to start with. However, you need to make a chat program first as that is more suitable for your initial task. Writing a non network game can be challenging enough if you are not a game programmer and have a lot of experience. Trying to write a networked game when you don't have network experience either is all the more harder!

Starting with a chat application (something like IRC setup more so than a p2p messenger) would help you focus on the networking first. You want to allow users to create channels, change their names, and ensure messages are able to be sent to all intended recipients. Once you get comfortable with the process, you can apply those concepts to your simple game

The reason you don't want to just go right into player movement in a world is because the concept of player movement in a game can be a lot more complex than desirable depending on what type of game it is. For the sake of just getting familiar with this stuff, implementing the movement in an unrealistic way won't help you much because it's simple message passing. That same concept of message passing is why a chat application is more suitable to start with.

Does anyone know where I can get specific information on how the object visit pattern works? I would like to learn more about it to get a more in-depth understanding of it. [/quote]

You won't really need any more information about the design itself; you have everything you need. However, it sounds like you need to learn quite a bit more C++ to really understand what is going on. What compiler and IDE (if any) are you working with now?

To summarize the design:

1. Write a "stream reader" and "stream writer" class. (Mine were called DeseralizeStream and SeralizeStream. hplus's were called InStream and presumably OutStream). These classes must implement 'visit' functions for each data type you wish to serialize to the stream. The concept of serializing data needs to take into account the endianness, so that is why there is so much code to support both little and big endian (it's effectively 2x the code). Most people just choose one format and stick with it, but being able to support both is a bonus.

2. Write free standing templated visit functions for each higher level object type you will be serializing. The function signature should be: "template< typename Stream >Stream & visit( YOURTYPE & , Stream & )". Now, inside this function, you simply call the parameter stream's visit function on all member variables that should be serialized. You choose which ones you need. The nice thing about this design (from some perspectives) is you get control over what gets serialized and what does not. Only the member variables that you explicitly call visit on will get visited!

At this point, you will now be able to serialize and deserialize objects into a blob of data. However, all you have is a "payload". You have to attach more accompanying data before sending it across the network so it's meaningful.

3. Implement your own network protocol (assuming you already have a network library/framework to use). This will allow you to take the payload of your messages (what the visit pattern gives you as output) and attach what type of data it is (usually designated by an 'opcode') as well as the size. Remember TCP is a stream, so you must send the size of the data that is following. Once you have your own protocol, you can now just communicate between endpoints and worry about the actual network logic. If you are using UDP, then everything stays the same, except your protocol is slightly different. However, that is a different topic, so in context of this thread, everything is the same in regards to the visit pattern.

That's literally all there is to it.

An analogy of how it works would be, consider using a fork and a spoon for a meal. That is the traditional method of having WriteType and ReadType functions in different sections of logic. The visitor pattern would be like using a spork. You still have your spoon and fork, it's just in one tool (thanks to how C++ works, via templates and overloading). In the end, the functionality is the same, data still has to get written and read, you just go about different ways to implement it (and each have their pros and cons).

If you were using a language that did not support function overloading or templates, then this type of design could not be as efficiently expressed in the language. You would end up with similar code, just a lot more of it (minus the benefits already discussed).

Anyways, hope that helps. Good luck with your learning!

##### Share on other sites
Sounds good. A simple tech demo of networked interactions is something that is pretty simple and good to start with. However, you need to make a chat program first as that is more suitable for your initial task. Writing a non network game can be challenging enough if you are not a game programmer and have a lot of experience. Trying to write a networked game when you don't have network experience either is all the more harder!
[/quote]

I am not sure writing a chat program is a suitable place for me to start. I did not to long ago create a client and server for sharing libraries. The server would allow multiple clients to connect and download files off the server. I am very familiar as well with networking. Last year I passed my CCNA course. I therefore know a great deal of how network protocol works as well as there structure.

Personally what I am building I would hardly count as a game. The graphics and mechanics for this "game" is the least thing I am concerned about. Its almost unimportant to an extent. Being able to build my own protocol and with flow control and etc is what's more important. My university course is called Computer systems and Networks. The only thing I am required to demonstrate for my final year project is some form of distributed system. So a chat program would do the trick, however I have just spent the last 7/8 months building the framework for a game engine. I don't really want to stop doing that just to take on another project. I still am open to that sugestion though.

I completely understand how the serialization works now after playing around quite a bit with hplus code (I could not get your posted code to work, I am sure I am missing something there).

[color="#1C2837"]
Implement your own network protocol (assuming you already have a network library/framework to use). This will allow you to take the payload of your messages (what the visit pattern gives you as output) and attach what type of data it is (usually designated by an 'opcode') as well as the size. Remember TCP is a stream, so you must send the size of the data that is following. Once you have your own protocol, you can now just communicate between endpoints and worry about the actual network logic. If you are using UDP, then everything stays the same, except your protocol is slightly different. However, that is a different topic, so in context of this thread, everything is the same in regards to the visit pattern. [/quote]

[color="#1C2837"]For the game I plan on using UDP, however if I was to go down the path of a chat program I would of course use TCP for the reliability sake. I am a bit unclear by what you mean when you say "opcode"? I have used hplus code to serialize all my data but can't figure out how to send it across a local network. I can send the data but it shows up on the other end as being the same size/capacity, it just do not get the data.

[color="#1C2837"]I understand how the serialization works but after using hplus code it seems to have all the bytes in a buffer as a char. When I go to deserialize this code how can you determine what the code was in its original state? eg an integer, char, string and etc....

[color="#1C2837"]I feel like this serialization stuff is a just a bit of a roadblock right now and that if I can get around it I would be set. I know thats not how these things always work. I thought about maybe using the boost library for this but heard it has a lot of overhead. What do you guys think about the boost library?

[color="#1C2837"]I know you probably have explained this all to me several times and i thank you for your patience. I am doing my best to achieve this myself.

[color="#1C2837"]Many thanks
[color="#1C2837"]agisler

[color="#1C2837"]p.s - I am using visual studio 2010 for my IDE.

##### Share on other sites

The world is changing. I've long thought for example, that Microsoft is at a disadvantage to Google. Google wasn't afraid of the cloud; they embraced it. Microsoft goes around trying to get countries to crack down on software theft. I think that's a tough proposition and they are huge. I don't have to worry about things like that with this model.

Here we go again.

http://online.wsj.co...p_sections_tech

The article is helpful, but would be even better if they would call a spade a spade. Let's drop the use of the word pirate. It is stealing. The Chinese lead the world in theft. Thank G-d for the cloud.

##### Share on other sites

I don't understand why you would want to be able to resend a packet when your running a real time game. Am I missing something here?

After looking over this discussion, I'd like to answer in a slightly different way:

If I send a "position update" message, and it gets dropped, then that doesn't matter so much, because I will send another, newer, position update message in a few dozen milliseconds anyway.
However, if I send a "chat text" message, and it gets dropped, then I really do want to have it re-tried, because I want everyone to be able to read what I'm trying to say.
Similarly, in games, there are many kinds of data (evolving state) that really are "freshness over all," and other kinds of data (edge-triggers, say) that need to be reliably delivered. For example "so-and-so won the game and it's now over" is a message you really don't want some clients to miss :-)

Game networking has some unique challenges, because latency is really important, and the kinds of data it needs to deal with is diverse. Any game networking protocol will have to solve a bunch of appication-specific problems. By contrast, most other network protocols (HTTP, RPC, IPTV, VoIP, etc) do not have all of these problems at the same time, and thus are generally simpler in design.
However, the corollary here is: If you don't understand what happens on the inside of games and simulations already, you really aren't in a good position to come up with a robust game networking protocol that solves the actual challenges involved. And, because the challenges and trade-offs are different for different game genres, you generally focus on one particular genre (FPS, MMORPG, RTS, turn-based, ...) rather than a Grand Unified Theory of Game Networking.

There exists a lot of research on game-style networking already. However, most of the published research is for protocols like DIS (which is robust, but not very efficient) or HLA (which is neither robust nor efficient). Meanwhile, the state of the art is pushed forward by game studios, who don't really publish peer reviewed papers, so you'll have to trawl the Internet for references. "Quake III Networking," "Source Engine Networking" and "Age of Empires Lock-step Networking" are good things to search for (check the FAQ, for example). However, if this is a research project, a literature overview probably also need to talk about DIS and HLA to start with, and explain what's wrong with those approaches.

##### Share on other sites
Thank you hplus. That all makes perfect sense.

I have been working on the serialization and have that completely understood now. The only thing i have been struggling with is how to send the data once it has been serialized?

Many thanks

##### Share on other sites

I have been working on the serialization and have that completely understood now. The only thing i have been struggling with is how to send the data once it has been serialized?

You send data using send() (for TCP sockets) or sendto() (for UDP sockets).

You typically frame the data into some kind of wrapping header. For TCP, this will include the size and type of each message, so that it can be decoded from the "stream" of bytes you get on the other end.

For UDP, this will typically include many messages in the same datagram packet, with some kind of header that contains a datagram sequence number, acknowledgement for previously received datagrams, and timing information.

If you want to build reliability vs non-reliability into your send semantics, you need to do it on top of UDP. Look for example at Enet for an existing, open source implementation.