5 replies to this topic

[xynapse]

This time it's all about packet reconstruction - as we all know, with TCP they come in order, but can be fragmented, so let's say we're sending two packets but we receive 3 on the server side due to fragmentation.


Every packet is encapsulated to some sort of structure, let's say our packets are:

MOVE_TO
int x,int y;

LOOK_AT
int objectid;

Of course each packet has it's length ( int ) and type ( int ) attached.


Now client sends both at once (it happens) to server,
but server receives half of MOVE_TO as one packet, and the rest of MOVE_TO + LOOK_AT in the second packet.

Of course the same situation can happen on client side, so that's why i am thinking globally here.

Now how do you guys approach this, some sort of "Inbox" mechanism, where after recv you push_back to client's recv_box
and analyze the list of them, or?

Any chance for pseudocode or already implemented working solution - so we all could have a look please?


thx!

[hplus0603]

Yes -- there is a single input buffer. All data that comes in is copied to this buffer. Then the receive handler figures out how many complete packets there are in the buffer, and dispatches those to the appropriate recipient. If you send length first, this is simple -- you need sizeof(length) + length bytes of data in the buffer for a packet to be complete.
If you want to avoid sending length first, then you need to figure out some way to make the receive handler know about how to calculate the lenght of all possible packet types. It's possible, but more of a design challenge to keep it clean and not degenerate into a mess of classes of spaghetti code.

[xynapse]

I don't think i get it - let's say there is a buffer per session, and i got a recv from client (session)
i pass what was received into this buffer ( of course packets have length encapsulated ( the better scenario ) )


When and how do i analyze that buffer later on?

Let's say this is what is happening:

Session is just a client connected to the server:

// Let's say packets are made like this:
struct tPacket
{
int iLen;
int iType;
<some payload>

};



// Receive
for(clients;...)
{
// receive from client to buffer
	 int iBytesReceived = recv(client,buffer);
  
// push what was received to clients buffer
	 client->PutData(buffer);

}
// Analyze - iterate through clients list and analyze their buffers
for(clients;...)
{
  
  // check if buffer isn't empty
  if(client->hasSomethingInBuffer())
  {
	// get buffer data
	buffer = client->Getbuffer();

	// go through the buffer and handle all completed packets
   // then remove handled packets from client buffer
   // we know the length of each packet, so we know how many bytes from the buffer are needed for each one of them
	for(bufferindex=0;...)
	{
	 [b] // how do you do that actually?[/b]

       handlepacket(..);
       clearhandledpacketfrombuffer(...)
	 }

      
  }

[Matias Goldberg]

Basically one approach could be like this (let''s say it's split in 3 packets, using TCP):

1. Packet arrived. We get the length. It's incomplete. Store it in a buffer and do nothing yet.
2. Packet arrived. We see our buffer contains something, so append the incoming data to it. It's incomplete. Do nothing yet.
3. Packet arrived. We see our buffer contains something, so append the incoming data to it. It's complete. Move this data to a "complete packets" queue. Is there any leftover? If so check the length and see if we have another full packet to put in the "complete" queue or whether it should be in the "incomplete" buffer. Finally tell the application we have received full packets and clear the "complete" queue.

[AllEightUp]

I'm possibly misunderstanding the problem but if you are sending via tcp, it's the same as with udp, just more simplified. Udp sends are "packets" but you generally encode multiple messages into those packets for bandwidth reasons. The way you do that is very simple: "length" <data>, "length" <data>. With linear tcp data, same thing just that you don't have to deal with out of order or missing items. The code is just:

recv( void* buffer, size_t size )
{
 .. copy data to current buffer ..
 while( buffersize >= sizeof( SizeType ) )
 {
   if( buffersize > SizeType( *(SizeType*)buffer )
    .. build message from buffer data ..
    post the message to be processed
    remove this messages data from the buffer
 }
  post read more api call
}

Using boost::asio it really is that simple. With other solutions such as LibEvent or custom solutions it is as complicated as the interface requires.

[xynapse]

Ok works like a charm, i've setup a one buffer per session into which i put recv data, and analyze later on.

Thx for your suggestions.