Jump to content

  • Log In with Google      Sign In   
  • Create Account

We're offering banner ads on our site from just $5!

1. Details HERE. 2. GDNet+ Subscriptions HERE. 3. Ad upload HERE.


[solved] Calling recv() in a loop


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
4 replies to this topic

#1 Mitchell314   Members   -  Reputation: 185

Like
0Likes
Like

Posted 14 December 2013 - 04:37 PM

When reading a tcp socket for arbitrarily sized packets, is it safe to keep calling recv() until it returns a negative number? For simplicity's sake, I'm assuming no other error is raised, just that return value of 0 means connection closed, and -1 means there's no data left to be read at the time.

 

For example, would the following code properly read all of the socket's contents? The other end of the connection is sending data in the format of "<text> \n <text> \n <text> \n .... <text> \n"

/* Make believe string structure */
struct resizableString string;

int bufferSize = 100;
int socket = connectToOtherHost();

while (-1){
    char *pBuffer = calloc(bufferSize);
    int bytesRead = recv(socket, pBuffer, bufferSize, 0);

    if (bytesRead == 0) {
        printf("Client closed connection\n");

    } else if (bytesRead >0) {
        /* Appends fix-length buffer to a string */
        appendBufferToString(string, pBuffer, bufferSize);

    } else {
        /* Done reading */
        free(pBuffer);
        break;
    }
}
parsePacket(&string);


Sponsor:

#2 hplus0603   Moderators   -  Reputation: 5692

Like
1Likes
Like

Posted 14 December 2013 - 09:15 PM

First, if you loop more than once, then you will leak a pBuffer per iteration except for the last.

Second, a recv value of 0 or -1 may indicate the connection has been closed in some way.

Third, recv() will block if there is no data waiting in the receive buffer. And, if a client crashes out or somehow becomes "non-nicely" disconnected, that means recv() will block forever (or until the connection times out, which will typically take hours.)

To know whether recv() will block or not, use the select() call. Generally, a networked program will replace whatever the main loop top-level thing is with select(). If there is a delay to wait for simulating or rendering, for example, then that delay is done using select(). If everything just runs as fast as possible, call select() with a 0 timeout. When data is available to be read, call recv() once on that socket, into whatever buffer you're currently working on filling. You may need to call recv() more than once to fill the buffer you want. Also, you may receive the end of one buffer, and the beginning of the next buffer, in the same call to recv().
enum Bool { True, False, FileNotFound };

#3 Mitchell314   Members   -  Reputation: 185

Like
0Likes
Like

Posted 15 December 2013 - 12:34 AM

Ah thanks. That code was made up for this thread for simplicity's sake, the actual code does use select() and handles the memory buffers. The strategy I'm using is to keep a linked list of buffers per connection, where each recv() reads data into a new buffer, and if any data is written it is added to the end of the linked list. The buffers are freed when the parser processes the list later. According to the man pages for recv(2):

 

 

All three routines return the length of the message on successful  completion.   If  a  message  is  too  long to fit in the supplied buffer, excess bytes may be discarded depending on the type of socket the message is received from.

 

 

But it doesn't say which. And since I've been having networking-related bugs, I want to make sure that I'm not accidentally dropping data while not being susceptible to buffer overflow bugs/exploits. Blocking isn't too much of an issue; the program is supposed to be a daemon anyways. The program is connecting to game server instances, which may stream small or large amounts of data, and connecting to admin tools, which stream short commands. However, there is no set packet size, so I want to know if I can play it safe by reading from a socket into a growing list of buffers until it is dry.



#4 Bacterius   Crossbones+   -  Reputation: 9262

Like
1Likes
Like

Posted 15 December 2013 - 01:27 AM


But it doesn't say which. And since I've been having networking-related bugs, I want to make sure that I'm not accidentally dropping data while not being susceptible to buffer overflow bugs/exploits. Blocking isn't too much of an issue; the program is supposed to be a daemon anyways. The program is connecting to game server instances, which may stream small or large amounts of data, and connecting to admin tools, which stream short commands. However, there is no set packet size, so I want to know if I can play it safe by reading from a socket into a growing list of buffers until it is dry.

 

TCP (SOCK_STREAM) never drops data from the internal buffer, and will only give you what you request, leaving the rest available for subsequent calls to recv (I think internally it asks the remote host to send the data again if it can't keep up). So in that case you don't need to worry about losing bytes. What you do need to worry about is your stream getting fragmented, so you need to maintain some state during the connection, e.g. "ok so I've received 50 bytes of this 100-byte string, I still need 50 bytes..", because simply checking recv()'s return value as an indicator that a send() call on the remote host has been received is unreliable - TCP is stream-oriented, so it's possible that a remote host sends "hello" and then "test", and you, in your recv() loop, receive first "he", then "l", then "lote" and finally "st" (*this is a simplification, in reality this exact scenario does not really happen because packets have a minimum size before being fragmented, but it occurs all the time for longer streams). You cannot use recv()'s return value as an indicator that all data has made it to you, unless the remote host immediately closes afterwards (and even then I am not sure - someone will have to confirm this but I believe it's possible for the socket to close while data is still in transit to you, so you need some synchronization).

 

On the other hand UDP (SOCK_DGRAM) gives you no guarantees. The packet may have been discarded anywhere between source and destination, and that includes your system's internal buffer if it happens to run out of space. That's part of the deal, and it is a reality you will have to accept if you do UDP networking.


The slowsort algorithm is a perfect illustration of the multiply and surrender paradigm, which is perhaps the single most important paradigm in the development of reluctant algorithms. The basic multiply and surrender strategy consists in replacing the problem at hand by two or more subproblems, each slightly simpler than the original, and continue multiplying subproblems and subsubproblems recursively in this fashion as long as possible. At some point the subproblems will all become so simple that their solution can no longer be postponed, and we will have to surrender. Experience shows that, in most cases, by the time this point is reached the total work will be substantially higher than what could have been wasted by a more direct approach.

 

- Pessimal Algorithms and Simplexity Analysis


#5 Mitchell314   Members   -  Reputation: 185

Like
0Likes
Like

Posted 15 December 2013 - 02:00 AM

Thanks, that clears up my question. The protocols I'm using are just based around arbitrarily-length string messages separated by delimiters; message size is not communicated by any process. I've been burnt quite badly in the past by assuming that TCP preserves packet boundaries, hence the rotating buffers. :P

 

I've just never seen anybody else do it that way.






Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS