TCP/IP buffering
Author
Currently I am in the process of developing a new backend server and a test client that emulates our flash client using Tcp/Ip at work. Everything works fine for a while but then the buffer starts getting multiple packets all at once, the server and test client are both developed in C# using the .NET 2.0 framework. I've tried several different things to prevent this buffering, disabling the Nagle algorithm, trying IOControl.Flush, disabling fragmentation and so on but nothing has solved the problem. Does anyone know of a solution?
[disclaimer: I am not 100% sure of what you are asking.]
That is how TCP works. It knows nothing about packets (that is for the IP layer). The data is treated as a continuous stream.
You can implement packets, by using some kind of formatted data stream. An example would be to send a byte count with every message.
You can easily break the stream back into messages like this.
That is how TCP works. It knows nothing about packets (that is for the IP layer). The data is treated as a continuous stream.
You can implement packets, by using some kind of formatted data stream. An example would be to send a byte count with every message.
You can easily break the stream back into messages like this.
Author
Yes I understand that, but I havet to keep backwards compatability with the old backend server. They used TCP/IP using xml packets, in order for my new backend to work with the flash client I have to keep the packets all the same, I have no problem in this area, its when the server starts buffering and sending multiple packets in one that problems occur. If you need more information please let me know. Currently I am using async sockets using BeginReceive and EndReceive.
So, something like this:
<packet> <data>foo</data></packet>--------<packet> <data>foo</data></packet>--------<packet> <data>foo</data></packet><packet> <data>foo</data></packet><packet> <data>foo</data></packet><packet> <data>foo</data></packet>---------Quote:Original post by xDS4Lx
They used TCP/IP using xml packets, in order for my new backend to work with the flash client I have to keep the packets all the same, I have no problem in this area, its when the server starts buffering and sending multiple packets in one that problems occur.
This problem can only be solved by the means that rip-off posted. You will have to literally process the stream and rip out a complete packet when it's there and pass it on. Then, you move the buffer contents forward and wait for the rest of the remaining packets. In doing this, you will always keep backwards compatibility with the server; that is how it also works!
Here is an example of how my system works, followed by an explanation of what goes on:
// User function called when a client sends data to the server. Should return -1 // if the stream of data is not processable (needs more data) or a value > 0 // that represents how much data was processed. int TCPOnProcessStream(edx::TCPClientInterface * client, WORD size, LPBYTE stream) { tPacketHeader * header = (tPacketHeader*)stream; int packet_size = 0; bool encrypted = false; if(header->size & 0x8000) { packet_size = HandShakeApi.blowfish.GetOutputLength((header->size & 0x7FFF) + 4) + 2; encrypted = true; } else { packet_size = header->size + 6; } // See if we have a full packet yet if(packet_size > size) return -1; // If the packet is encrypted, fix the size and decrypt it if(encrypted) { header->size &= 0x7FFF; HandShakeApi.blowfish.Decode(stream + 2, stream + 2, packet_size - 2); } // Copy the entire packet into our buffer memcpy(streamBuffer, stream, header->size + 6); // Call the function to process the packet OnServerToClient(header, stream + 6); // Returns how many bytes were processed return packet_size; }The work flow of my library is as follows:
- Initialize Winsock/TCP Server/etc..
- curPos = 0
1- Recv at index "curPos" in buffer "recvBuffer".
2- curPos += the amount of data from Recv
3- call TCPOnProcessStream with "recvBuffer" and "curPos".
4- if TCPOnProcessStream returns -1, then goto 1
5- else curPos -= return value from TCPOnProcessStream.
6- if curPos != 0, memmove curPos bytes from recvBuffer + return value from TCPOnProcessStream into index 0 (move unprocessed data into buffer start)
7- goto 3 (see if there are more packets to process in the stream!)
This means that if you get some stream of data: AAABBBBBCCCCCCC in which A's are one packet, B's are a second packet, and C's are a third packet, the A packet will be the first to be extracted since the packet of A's was determined to be of size 3 and that is passed on. The same goes for B and finally C. Even though you got 3 packets at once, your code should break it down into 3 separate packets.
Aside from that, given the way TCP works, there is nothing else you can do to make sure you just get one packet at a time. Your old server client code at work should be designed to take that into account; if it doesn't, "big oops"! Hope that helps some. What does your xml packet format look like? Do you know the size of the data that is about to be received?
Author
Some of the packets we can determine the size ahead of time, but there are quite a few that have quite a few optional parameters so their size can not be gaurenteed ahead of time. The old server was written in C and flushed the buffer after the packet was sent their code is so horrible that it was decided to re-write the whole thing, I can't change any of the packets because I don't have the time to go back and change the flash client, we are on a strict deadline to get the new system in place since the old one is crapping out. My new server works fine when using the flash client ut my test windows client that emulates the flash client starts getting buffer problem. All of the packets are null terminated like c strings, however once the messages start to get buffered the entire data does not always end up in the packet.
Quote:Original post by xDS4Lx
All of the packets are null terminated like c strings, however once the messages start to get buffered the entire data does not always end up in the packet.
Using my example above, your packet logic would look like this in my code:
int TCPOnProcessStream(edx::TCPClientInterface * client, WORD size, LPBYTE stream){ for(int x = 0; x < size; ++x) { if(stream[x] == 0) { // // Process stream[0] -> stream[x] as one packet // // We processed exactly x bytes, if there is data from x+1 -> size, it will be called // on this function again. return x; } } return -1;}If you setup your recv code to make sure you are not recv'ing into index 0 all the time, which is what I think he problem could be, and instead recv at the 'curPos' index, you should be able to process packets fine without any problems. Make sure you move unprocessed data in the stream to the front of the buffer so it is not lost. What you are describing as a problem are things I ran into when setting up my logic. At times I was not copying the unprocessed data and ended up with packets lost and misparsed. My current approach works great though, so just passing along my experience.
Author
Quote:
If you setup your recv code to make sure you are not recv'ing into index 0 all the time, which is what I think he problem could be, and instead recv at the 'curPos' index, you should be able to process packets fine without any problems. Make sure you move unprocessed data in the stream to the front of the buffer so it is not lost. What you are describing as a problem are things I ran into when setting up my logic. At times I was not copying the unprocessed data and ended up with packets lost and misparsed. My current approach works great though, so just passing along my experience.
Thanks for the suggestion, I will try this tomorrow when I go into the office.
Author
Unfortunetly I wasnt able to get your solution to work as I hoped, after doing some more research I have found several suggestions of using NetworkStream, StreamWriter and StreamReader, however I havent been able to find any examples that are helpful.
Quote:Original post by xDS4Lx
Unfortunetly I wasnt able to get your solution to work as I hoped, after doing some more research I have found several suggestions of using NetworkStream, StreamWriter and StreamReader, however I havent been able to find any examples that are helpful.
Ah sorry to hear. Here's an example of what I do on the backend since I only showed a frontend example. This is in C++, so not sure of the conversions you have to do to make it work properly in C# in regards to arrays and what not:
// The largest size a complete packet can be, used for recv() as well as// the stream parsing buffer size#define MAX_PACKET_SIZE 8192...// This object's socketSOCKET sSocket;// Main recv bufferCHAR * recvBuffer; // allocated with MAX_PACKET_SIZE bytes// Recv buffer work spaceCHAR * localRecvBuffer; // allocated with MAX_PACKET_SIZE bytes// Index the recv buffer is atint recvIndex; // = 0 to start...// Called when a client sends data to the servervoid edx::TCPClientInterface::OnRead(){ // Recv index INT recvCount = 0; // Recv data - only enough to fill the recv buffer, not more recvCount = recv(sSocket, recvBuffer, MAX_PACKET_SIZE - recvIndex, 0); // Make sure there was not an error if(recvCount != SOCKET_ERROR) { // Copy it into the building buffer, recvIndex will never be > EDX_Max_Recv_Size // if the library is used correctly. memcpy(localRecvBuffer + recvIndex, recvBuffer, recvCount); // Increase the index recvIndex += recvCount; // Call the user function to process the stream, should return -1 if there is not a full packet yet // or any number that tells how many bytes were processed INT result = this->parentInterface->TCPOnProcessStream(this, (WORD)recvIndex, (LPBYTE)localRecvBuffer); // Loop while we have a full packet while(result != -1) { // Decrease buffer size recvIndex -= result; // Physically overwrite existing data, we have to do this, no way around it without // making things way more complicated. By doing this also, we guarantee that we will // never overflow our packet buffer if everything is used correctly. if(recvIndex) { // Move the data that is left into the start of the buffer memmove(localRecvBuffer, localRecvBuffer + result, recvIndex); // Call the user function to process the stream, should return -1 if there is not a full packet yet // or any number that tells how many bytes were processed result = this->parentInterface->TCPOnProcessStream(this, (WORD)recvIndex, (LPBYTE)localRecvBuffer); } else { // If there is no data left, then we do not need to call UserProcessStream, // so we are done in this loop break; } } }}One important thing to note is that the largest complete packet is 8kb. This means that no packet can ever span outside of the current recv buffer. The way my logic works is, by making the largest packet size a fixed value, no additional de/allocations are ever needed in the packet processing logic since the call to memmove will move existing data into the front of the buffer. The next call to recv will only fill up to the max packet's size (MAX_PACKET_SIZE - recvIndex).
Of course, this all works if the end user correctly returns the number of bytes processed from TCPOnProcessStream. However, I place good confidence in the users to do so [wink] Hope this helps explaining my ideas, good luck solving your problem! I would also suggest making sure you aren't suffering from any unfortunate side affects from whatever C# constructs you are using to do networking.
I don't have a specific example for C#, but in C++, if you used strlen to get your packet size based on text, you won't be accounting for any data past the '\0', so using strlen to get packet size might be misleading as to how much text there is to process (silly example, but it could happen). Kinda see what I mean?
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
