Server Design and Protocol
Author
Hi, I want to start out with a basic chat server with a few commands. If anybody knows of a small, to the point, example (Preferably in C) of such a thing, that would be great.
My first question is how to properly make use of packets with the header and data? Example:
Packet type (int) | Packet length (int) | Packet data
I'm not sure exactly how to implement protocols like this. Would you copy the incoming buffer area into a new packet struct when the buffer length is header size plus packet length, and then work with it there? I think this would be usefull to use as shared packets (next paragraph).
If say five people were in a chat room, would you want to make a shared packet to reduce memory copying, or just copy the data into all of the user's buffers? I'm also not sure of the best way to do this...
I'm also planning on releasing the source code when I'm done, in order to help other new programmers. A lot of examples I see don't even check to see how many bytes were sent or received, and no buffers, so I think it would be useful.
I hope that makes some sense about what kind of problems I'm having. Any help would be appreciated.
A lot of the time, it's simpler just to make a packet for everybody, and track each packet individually.
One way that you can approach multi-target packets is by including a target structure pointer and count in the packet (or vector), and have the data as a self-deleting refcounted object, not part of the packet wrapper.
Something along those lines anyway.
One way that you can approach multi-target packets is by including a target structure pointer and count in the packet (or vector), and have the data as a self-deleting refcounted object, not part of the packet wrapper.
[source=cpp]struct Target{ IPAddress address; // Other target information - client ID?}struct PacketContents : public RefCountObject // Only delete on zero refcount{ unsigned short length; unsigned char data[0]; // Zero length array marks data block.}struct PacketWrapper{ std::vector<Target>; // cpp version Target* pTarget; unsigned int m_iNumTargets; PacketContents* m_pContents;};Something along those lines anyway.
I assume this is for TCP, as in UDP you don't have to worry about your packet being the right length. For the packetising of TCP stream data, check out my CodeProject article and source here. The salient part is the ReadInternal method:
... and the FillHeader method:
void ReadInternal(byte[] buf, int read){ if((OnReadMessage != null) && (MessageType != MessageType.Unmessaged)){ // Messaged mode int copied; uint code = 0; switch(MessageType){ case MessageType.CodeAndLength: case MessageType.Length: int length; if(MessageType == MessageType.Length){ copied = FillHeader(ref buf, 4, read); if(headerread < 4) break; length = GetInt(msgheader, 0, 4); } else{ copied = FillHeader(ref buf, 8, read); if(headerread < 8) break; code = (uint)GetInt(msgheader, 0, 4); length = GetInt(msgheader, 4, 4); } bytes.Add(buf, 0, read - copied); if(bytes.Length >= length){ // A message was received! headerread = 0; byte[] msg = bytes.Read(0, length); OnReadMessage(this, code, msg, length); // Don't forget to put the rest through the mill byte[] whatsleft = bytes.Read(length, bytes.Length - length); bytes.Clear(); if(whatsleft.Length > 0) ReadInternal(whatsleft, whatsleft.Length); } //if(OnStatus != null) OnStatus(this, bytes.Length, length); break; } } }... and the FillHeader method:
int FillHeader(ref byte[] buf, int to, int read) { int copied = 0; if(headerread < to){ // First copy the header into the header variable. for(int i = 0; (i < read) && (headerread < to); i++, headerread++, copied++){ msgheader[headerread] = buf; } } if(copied > 0){ // Take the header bytes off the 'message' section byte[] newbuf = new byte[read - copied]; for(int i = 0; i < newbuf.Length; i++) newbuf = buf;<br> buf = newbuf;<br> }<br> <span class="cpp-keyword">return</span> copied;<br> }<br></pre></div><!–ENDSCRIPT–><br><br>In TCP, you would have one of the users being a server through which all messages were sent, which would then broadcast it to the others.
Author
Thanks for the help guys. Although would the way to do this be best for copying data from the recv's buffer into a packet struct using memcpy, then using memmove to shift anything extra in the buffer (that might be part of an incomplete packet)?
Bob Janova: I'm having a little trouble following your code, I've never done anything in C#. In your artical though, your description of big endian seems a little strange.
"In the current version, a 4-byte length parameter is sent before the data (big-endian); for example, a message containing the word "Bob" would have the byte form 00 00 00 03 42(B) 6F(o) 62(b)."
Wouldn't Bob be the same in big or little endian? 00 00 00 03 would change to 03 00 00 00. Even if for some reason you sent the byte length after the data, that wouldn't change big/little endian, right?
Bob Janova: I'm having a little trouble following your code, I've never done anything in C#. In your artical though, your description of big endian seems a little strange.
"In the current version, a 4-byte length parameter is sent before the data (big-endian); for example, a message containing the word "Bob" would have the byte form 00 00 00 03 42(B) 6F(o) 62(b)."
Wouldn't Bob be the same in big or little endian? 00 00 00 03 would change to 03 00 00 00. Even if for some reason you sent the byte length after the data, that wouldn't change big/little endian, right?
One example is Etwork which comes with a chat client and server application. Unfortunately, it uses C++ abstract base classes to isolate a number of interfaces, rather than C, but it might be worth a shot. Warning: the samples require Windows!
send() /sendto() will tell you the number of bytes sent.
Simply do not deallocate or free the packet being sent until it is sent in full,
and retry sending it when the transmission buffer is empty.
Strings are not endian - they are usually sent as plain C strings (zero terminated). The integer length would change - a message saying 'bob' in little-endian would be: 04 00 00 00 42 6f 62 00
More complex message headers are usually used - a message type, ID etc, hash or checksum as well as length, to allow a packet to carry the ID of the message it is part of, as well as a packet payload length and offset into the message.
Messages should be constructed from packets. Although TCP is a stream protocol, it still only has finite buffer size, which is partially freed when the other side recv()'s, and the packet sent is acknowledged. It's a good idea to implement a packetising system to chop the message up, even if the majority of your messages will end up as a single packet - it saves the headache later.
You might do something like this:
If you send a maximum packet payload of 4 bytes (for example) you'd send the following packets, and acknowledge / resend as required:
You can see how horribly inefficient this is (just look at all those zeroes) - if you're sending nothing but small packets, you don't need to use ints as sizes and offsets, when a byte will do. Implementing this kind of system is a good step toward reliable UDP (it just becomes an excercise in packet queuing) but also helps tackle truncated TCP sends, since only unsent portions of the message (not the whole thing) need to be kept around.
Simply do not deallocate or free the packet being sent until it is sent in full,
and retry sending it when the transmission buffer is empty.
Strings are not endian - they are usually sent as plain C strings (zero terminated). The integer length would change - a message saying 'bob' in little-endian would be: 04 00 00 00 42 6f 62 00
More complex message headers are usually used - a message type, ID etc, hash or checksum as well as length, to allow a packet to carry the ID of the message it is part of, as well as a packet payload length and offset into the message.
Messages should be constructed from packets. Although TCP is a stream protocol, it still only has finite buffer size, which is partially freed when the other side recv()'s, and the packet sent is acknowledged. It's a good idea to implement a packetising system to chop the message up, even if the majority of your messages will end up as a single packet - it saves the headache later.
You might do something like this:
Message:00 00 00 01 : int - message sequence number00 00 00 04 : int - message payload length 00 : byte- message type00 : byte- message flags - include a 'multi-packet' flag00 00 01 13 : int - simple checksum of payload------------: unsigned char[] - message payload (variable size)"Bob" : zero-terminated string (4 bytes)If you send a maximum packet payload of 4 bytes (for example) you'd send the following packets, and acknowledge / resend as required:
Packet:00 00 00 01 : int - message sequence number00 00 00 00 : int - offset into message (INCLUDE HEADER!)00 00 00 04 : int - packet payload length00 : byte - packet flags (priority etc)------------: unsigned char[] - packet payload00 00 00 01 (message ID)Packet:00 00 00 01 : int - message sequence number00 00 00 04 : int - offset into message00 00 00 04 : int - packet payload length00 : byte - packet flags (priority etc)------------: unsigned char[] - packet payload00 00 00 04 (message payload length)Packet:00 00 00 01 : int - message sequence number00 00 00 08 : int - offset into message00 00 00 04 : int - packet payload length00 : byte - packet flags (priority etc)------------: unsigned char[] - packet payload00 00 00 00 : (type, flags, first 2 bytes of checksum)Packet:00 00 00 01 : int - message sequence number00 00 00 0C : int - offset into message00 00 00 04 : int - packet payload length00 : byte - packet flags (priority etc)------------: unsigned char[] - packet payload01 13 42 6f : (last 2 bytes of checksum, 'B', 'o')Packet:00 00 00 01 : int - message sequence number00 00 00 10 : int - offset into message00 00 00 02 : int - packet payload length00 : byte - packet flags (priority etc)------------: unsigned char[] - packet payload62 00 : ('b', zero)You can see how horribly inefficient this is (just look at all those zeroes) - if you're sending nothing but small packets, you don't need to use ints as sizes and offsets, when a byte will do. Implementing this kind of system is a good step toward reliable UDP (it just becomes an excercise in packet queuing) but also helps tackle truncated TCP sends, since only unsent portions of the message (not the whole thing) need to be kept around.
Quote:Wouldn't Bob be the same in big or little endian? 00 00 00 03 would change to 03 00 00 00. Even if for some reason you sent the byte length after the data, that wouldn't change big/little endian, right?
Yes, the endian-ness applies to the 3, not the ordering of the data. The exact details of how I do the parameterisation aren't important, really. And I apologise for giving you C# code; I hadn't read that you were writing this in C! The basic idea of adding a length-check to the front of the message will be the same. I'd suggest something like this (pseudo-code, as I don't know how C does network stuff):
int read = 0;byte header[4];// Read the header: keep asking for more data until we have// 4 bytes total. This will block, so if you're in a Windoze// app it should be in a second thread. Need a recv() function// taking the target buffer and max length to readwhile(true){ int got = recv(header[read], 4 - read)); if(got < 0){ printf(stderr, "Read failed"); return; } read += got; if(read >= 4) break;}int len = *header; // does this work?read = 0;byte *buf = alloc(len);// Now read the message body, as above while(read < len){ int got = recv(buf[read], len - read)); if(got < 0){ printf(stderr, "Read failed"); return; } read += got;}// Do stuff with buffree(buf);That may be all uncompilable, it's ages since I did any C, but hopefully you get the idea.
Hashing and checksums are not required for packetising TCP as the protocol itself guarantees that what you see is complete and uncorrupted.
Edit: The content of the message may well contain information that can be mapped to a struct of some kind, which you may want to return from the function. I.e.
typedef struct { int ID, type; byte content;} message_t;...// at the end of the function above, instead of free(buf) return (message_t*)buf;
Author
Thanks for the good replies everybody. Especially _winterdyne_, you've been a lot of help :)
luzarius: there is a good topic on this forum - Input/Output Processing From Sockets - mainly about the circular/ring buffer. That is if you're watching this topic since you've shown interest.
Bob Janova: I think a better way of doing that is just recv and add to the buffer, and whenever anything is added, check to see if it makes a complete packet. That way you're not blocking while other clients might be trying to send.
luzarius: there is a good topic on this forum - Input/Output Processing From Sockets - mainly about the circular/ring buffer. That is if you're watching this topic since you've shown interest.
Bob Janova: I think a better way of doing that is just recv and add to the buffer, and whenever anything is added, check to see if it makes a complete packet. That way you're not blocking while other clients might be trying to send.
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
