Sign in to follow this  
xrazybud

Server Design and Protocol

Recommended Posts

Hi, I want to start out with a basic chat server with a few commands. If anybody knows of a small, to the point, example (Preferably in C) of such a thing, that would be great. My first question is how to properly make use of packets with the header and data? Example: Packet type (int) | Packet length (int) | Packet data I'm not sure exactly how to implement protocols like this. Would you copy the incoming buffer area into a new packet struct when the buffer length is header size plus packet length, and then work with it there? I think this would be usefull to use as shared packets (next paragraph). If say five people were in a chat room, would you want to make a shared packet to reduce memory copying, or just copy the data into all of the user's buffers? I'm also not sure of the best way to do this... I'm also planning on releasing the source code when I'm done, in order to help other new programmers. A lot of examples I see don't even check to see how many bytes were sent or received, and no buffers, so I think it would be useful. I hope that makes some sense about what kind of problems I'm having. Any help would be appreciated.

Share this post


Link to post
Share on other sites
A lot of the time, it's simpler just to make a packet for everybody, and track each packet individually.

One way that you can approach multi-target packets is by including a target structure pointer and count in the packet (or vector), and have the data as a self-deleting refcounted object, not part of the packet wrapper.

[source=cpp]
struct Target
{
IPAddress address;
// Other target information - client ID?
}

struct PacketContents : public RefCountObject // Only delete on zero refcount
{
unsigned short length;
unsigned char data[0]; // Zero length array marks data block.
}

struct PacketWrapper
{
std::vector<Target>; // cpp version

Target* pTarget;
unsigned int m_iNumTargets;

PacketContents* m_pContents;
};




Something along those lines anyway.

Share this post


Link to post
Share on other sites
I assume this is for TCP, as in UDP you don't have to worry about your packet being the right length. For the packetising of TCP stream data, check out my CodeProject article and source here. The salient part is the ReadInternal method:
  void ReadInternal(byte[] buf, int read){
if((OnReadMessage != null) && (MessageType != MessageType.Unmessaged)){
// Messaged mode
int copied;
uint code = 0;
switch(MessageType){
case MessageType.CodeAndLength:
case MessageType.Length:
int length;
if(MessageType == MessageType.Length){
copied = FillHeader(ref buf, 4, read);
if(headerread < 4) break;
length = GetInt(msgheader, 0, 4);
} else{
copied = FillHeader(ref buf, 8, read);
if(headerread < 8) break;
code = (uint)GetInt(msgheader, 0, 4);
length = GetInt(msgheader, 4, 4);
}
bytes.Add(buf, 0, read - copied);
if(bytes.Length >= length){
// A message was received!
headerread = 0;
byte[] msg = bytes.Read(0, length);
OnReadMessage(this, code, msg, length);
// Don't forget to put the rest through the mill
byte[] whatsleft = bytes.Read(length, bytes.Length - length);
bytes.Clear();
if(whatsleft.Length > 0) ReadInternal(whatsleft, whatsleft.Length);
}
//if(OnStatus != null) OnStatus(this, bytes.Length, length);
break;
}
}
}

... and the FillHeader method:
  int FillHeader(ref byte[] buf, int to, int read) {
int copied = 0;
if(headerread < to){
// First copy the header into the header variable.
for(int i = 0; (i < read) && (headerread < to); i++, headerread++, copied++){
msgheader[headerread] = buf[i];
}
}
if(copied > 0){
// Take the header bytes off the 'message' section
byte[] newbuf = new byte[read - copied];
for(int i = 0; i < newbuf.Length; i++) newbuf[i] = buf[i + copied];
buf = newbuf;
}
return copied;
}


In TCP, you would have one of the users being a server through which all messages were sent, which would then broadcast it to the others.

Share this post


Link to post
Share on other sites
Thanks for the help guys. Although would the way to do this be best for copying data from the recv's buffer into a packet struct using memcpy, then using memmove to shift anything extra in the buffer (that might be part of an incomplete packet)?

Bob Janova: I'm having a little trouble following your code, I've never done anything in C#. In your artical though, your description of big endian seems a little strange.

"In the current version, a 4-byte length parameter is sent before the data (big-endian); for example, a message containing the word "Bob" would have the byte form 00 00 00 03 42(B) 6F(o) 62(b)."

Wouldn't Bob be the same in big or little endian? 00 00 00 03 would change to 03 00 00 00. Even if for some reason you sent the byte length after the data, that wouldn't change big/little endian, right?

Share this post


Link to post
Share on other sites
One example is Etwork which comes with a chat client and server application. Unfortunately, it uses C++ abstract base classes to isolate a number of interfaces, rather than C, but it might be worth a shot. Warning: the samples require Windows!

Share this post


Link to post
Share on other sites
send() /sendto() will tell you the number of bytes sent.

Simply do not deallocate or free the packet being sent until it is sent in full,
and retry sending it when the transmission buffer is empty.

Strings are not endian - they are usually sent as plain C strings (zero terminated). The integer length would change - a message saying 'bob' in little-endian would be: 04 00 00 00 42 6f 62 00

More complex message headers are usually used - a message type, ID etc, hash or checksum as well as length, to allow a packet to carry the ID of the message it is part of, as well as a packet payload length and offset into the message.

Messages should be constructed from packets. Although TCP is a stream protocol, it still only has finite buffer size, which is partially freed when the other side recv()'s, and the packet sent is acknowledged. It's a good idea to implement a packetising system to chop the message up, even if the majority of your messages will end up as a single packet - it saves the headache later.

You might do something like this:


Message:
00 00 00 01 : int - message sequence number
00 00 00 04 : int - message payload length
00 : byte- message type
00 : byte- message flags - include a 'multi-packet' flag
00 00 01 13 : int - simple checksum of payload
------------: unsigned char[] - message payload (variable size)
"Bob" : zero-terminated string (4 bytes)



If you send a maximum packet payload of 4 bytes (for example) you'd send the following packets, and acknowledge / resend as required:


Packet:
00 00 00 01 : int - message sequence number
00 00 00 00 : int - offset into message (INCLUDE HEADER!)
00 00 00 04 : int - packet payload length
00 : byte - packet flags (priority etc)
------------: unsigned char[] - packet payload
00 00 00 01 (message ID)


Packet:
00 00 00 01 : int - message sequence number
00 00 00 04 : int - offset into message
00 00 00 04 : int - packet payload length
00 : byte - packet flags (priority etc)
------------: unsigned char[] - packet payload
00 00 00 04 (message payload length)


Packet:
00 00 00 01 : int - message sequence number
00 00 00 08 : int - offset into message
00 00 00 04 : int - packet payload length
00 : byte - packet flags (priority etc)
------------: unsigned char[] - packet payload
00 00 00 00 : (type, flags, first 2 bytes of checksum)


Packet:
00 00 00 01 : int - message sequence number
00 00 00 0C : int - offset into message
00 00 00 04 : int - packet payload length
00 : byte - packet flags (priority etc)
------------: unsigned char[] - packet payload
01 13 42 6f : (last 2 bytes of checksum, 'B', 'o')


Packet:
00 00 00 01 : int - message sequence number
00 00 00 10 : int - offset into message
00 00 00 02 : int - packet payload length
00 : byte - packet flags (priority etc)
------------: unsigned char[] - packet payload
62 00 : ('b', zero)



You can see how horribly inefficient this is (just look at all those zeroes) - if you're sending nothing but small packets, you don't need to use ints as sizes and offsets, when a byte will do. Implementing this kind of system is a good step toward reliable UDP (it just becomes an excercise in packet queuing) but also helps tackle truncated TCP sends, since only unsent portions of the message (not the whole thing) need to be kept around.










Share this post


Link to post
Share on other sites
Quote:
Wouldn't Bob be the same in big or little endian? 00 00 00 03 would change to 03 00 00 00. Even if for some reason you sent the byte length after the data, that wouldn't change big/little endian, right?


Yes, the endian-ness applies to the 3, not the ordering of the data. The exact details of how I do the parameterisation aren't important, really. And I apologise for giving you C# code; I hadn't read that you were writing this in C! The basic idea of adding a length-check to the front of the message will be the same. I'd suggest something like this (pseudo-code, as I don't know how C does network stuff):

int read = 0;
byte header[4];

// Read the header: keep asking for more data until we have
// 4 bytes total. This will block, so if you're in a Windoze
// app it should be in a second thread. Need a recv() function
// taking the target buffer and max length to read
while(true){
int got = recv(header[read], 4 - read));
if(got < 0){ printf(stderr, "Read failed"); return; }
read += got;
if(read >= 4) break;
}

int len = *header; // does this work?
read = 0;
byte *buf = alloc(len);

// Now read the message body, as above
while(read < len){
int got = recv(buf[read], len - read));
if(got < 0){ printf(stderr, "Read failed"); return; }
read += got;
}

// Do stuff with buf

free(buf);



That may be all uncompilable, it's ages since I did any C, but hopefully you get the idea.

Hashing and checksums are not required for packetising TCP as the protocol itself guarantees that what you see is complete and uncorrupted.

Edit: The content of the message may well contain information that can be mapped to a struct of some kind, which you may want to return from the function. I.e.
typedef struct {
int ID, type;
byte content;
} message_t;

...
// at the end of the function above, instead of free(buf)
return (message_t*)buf;

Share this post


Link to post
Share on other sites
Thanks for the good replies everybody. Especially _winterdyne_, you've been a lot of help :)

luzarius: there is a good topic on this forum - Input/Output Processing From Sockets - mainly about the circular/ring buffer. That is if you're watching this topic since you've shown interest.

Bob Janova: I think a better way of doing that is just recv and add to the buffer, and whenever anything is added, check to see if it makes a complete packet. That way you're not blocking while other clients might be trying to send.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this