Jump to content

  • Log In with Google      Sign In   
  • Create Account


recv() stops when there's exactly 508 bytes left.


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
17 replies to this topic

#1 Zomgbie   Members   -  Reputation: 236

Like
0Likes
Like

Posted 24 February 2011 - 06:37 AM

Hello you!

As the topic states, my recv() call keeps stopping when there is 508 bytes left to read. I'm trying to send a file, and it's all going good untill we hit the magical 508 bar.

Sending

void
sendFile(char *fileName)
{
        struct fileInfo 		info;
        FILE                    *file;
        long                    fileSize;
        char                    *buffer, temp[512];
        int         			left, sent, total;

        /* Open the file */
        file = fopen(fileName, "rb");
        if (file == NULL)
                logThis("sendFile", "Out of memory while opening file.");

        /* Obtain file zie */
        fseek(file, 0, SEEK_END);
        fileSize = ftell(file);
        rewind(file);

        /* Allocate memory for the buffer */
        buffer = (char *)malloc(sizeof(char) * fileSize);
        if (buffer == NULL) {
                logThis("sendFile", "Out of memory.");
                exit(99);
        }

        /* Copy file contents over to our buffer */
        fread(buffer, 1, fileSize, file);
        logThis("sendFile", "Buffer contains: %s", buffer);

        /* Send the file name first, and then our size */
        info.size = fileSize;
        snprintf(info.name, 32, "%s", fileName);
        logThis("sendFile", "Name: %s, size: %d", info.name, info.size);
        sendCommand(NET_FILEINFO, (char *)&info);

        /* Start sending file */
        total = 0;
        left = fileSize;
        logThis("sendFile", "Ready to send: %d bytes", left);
        while (total < fileSize) {
                sent = send(serverSocket, buffer + total, left, 0);


                if (sent == 0) {
                        logThis("handle", "Connection closed during transfer");
                        break;
                } else if (sent < 0) {
                        logThis("handle", "Error during tansfer!");
                        break;
                } else {
			total += sent;
                	left -= sent;
                	logThis("sendFile", "Sent: %d bytes. %d bytes left", sent, left);
		}
        }
        logThis("sendFile", "Done sending. Sent: %d vs %d", total, fileSize);
}

And where i get the file

void
*handle(void *argument)
{
        struct fileClient   	*client;
        struct fileInfo 		*info;
        char                    buffer[512], *fileBuffer;
        FILE                    *file;
        int         			total, read, ret;

        /* Get the client */
        client = (struct fileClient *)argument;

        /* Ready for files, send OK message */
        sendCommand(client->clientSocket, NET_OK, " ");

        /* Expect file name and then size */
        recvCommand(client->clientSocket, NET_FILEINFO, buffer);
        info = (struct fileInfo *)buffer;
        logThis("handle", "File name: %s file size: %d", info->name, info->size);

        /* Allocate memory for the file */
        fileBuffer = (char *)malloc(info->size);

        /* Keep recieving data and copying it into the file buffer untill done */
        total = 0;
        read = info->size;
        while (read > 0) {
                ret = recv(client->clientSocket, (fileBuffer + total), read, 0);

                if (ret == 0) {
                        logThis("handle", "Connection closed during transfer");
                        break;
                } else if (ret < 0) {
                        logThis("handle", "Error during tansfer!");
                        break;
                } else {
                        total += ret;
                        read -= ret;
                        logThis("handle", "Read: %d bytes. %d bytes left", ret, read);
                }
        }

        /* Create the file on disk and save it */
        file = fopen(info->name, "wb");
        fwrite(fileBuffer, 1, info->size, file);
        fclose(file);
        pthread_exit(NULL);
}

I should note that sometimes, the file manages to be downloaded completly.
Omg, zombie! Zomgbie.

Sponsor:

#2 Erik Rufelt   Crossbones+   -  Reputation: 3171

Like
0Likes
Like

Posted 24 February 2011 - 07:06 AM

Your while(total < left) loop will be wrong when you both increase total and decrease left, as they will meet on the middle. You probably want while(total < filesize).

#3 Zomgbie   Members   -  Reputation: 236

Like
0Likes
Like

Posted 24 February 2011 - 07:24 AM

You're correct. Not sure what i missed in my calculations, but indeed, it will skip when they meet on the middle.

But i still have the exact same problem. My send() function reports that all data have been sent, but when the recv() functions hits 508 bytes, it just stops :(
Omg, zombie! Zomgbie.

#4 Erik Rufelt   Crossbones+   -  Reputation: 3171

Like
0Likes
Like

Posted 24 February 2011 - 08:01 AM

Change your client loop like this, and see if you get some new output.
ret = recv(client->clientSocket, (fileBuffer + total), read, 0);
if(ret == 0)
 // connection closed
else if(ret < 0)
 // error
else {
 total += ret;
 read -= ret;
}

Also, print out (or check with the debugger) the first 50 bytes or something of the file, to make sure they are equal on the server and client-side, so you don't miss any data at the start.

#5 Zomgbie   Members   -  Reputation: 236

Like
0Likes
Like

Posted 24 February 2011 - 08:08 AM

Nothing new sadly.

When there is 508 bytes left to read, recv just stops receiving. It will block the flow of the program (I'm using blocking sockets) untill the poor server shuts down, and therefore disconnects the client. There are no errors, and ret == 0 does not happend untill i actually shutdown the server. Feeling like i've gotten something very wrong, somewhere!

The output
...

[backupTool][2011-02-24][14:59:29][handle]: Read: 16384 bytes. 96348 bytes left
[backupTool][2011-02-24][14:59:29][handle]: Read: 16384 bytes. 79964 bytes left
[backupTool][2011-02-24][14:59:29][handle]: Read: 16384 bytes. 63580 bytes left
[backupTool][2011-02-24][14:59:29][handle]: Read: 16384 bytes. 47196 bytes left
[backupTool][2011-02-24][14:59:29][handle]: Read: 16384 bytes. 30812 bytes left
[backupTool][2011-02-24][14:59:29][handle]: Read: 16384 bytes. 14428 bytes left
[backupTool][2011-02-24][14:59:29][handle]: Read: 13920 bytes. 508 bytes left
... Nothing happens here, untill i disconnect the server. Then it says:

[backupTool][2011-02-24][14:59:40][handle]: Connection closed during transfer

Omg, zombie! Zomgbie.

#6 Erik Rufelt   Crossbones+   -  Reputation: 3171

Like
0Likes
Like

Posted 24 February 2011 - 08:27 AM

Did you compare the data you actually receive with the data you send on the server?
Try printing out the last 10 bytes sent from the server and the last 10 bytes received on the client, to make sure your data isn't just offset 508 bytes, and the last byte you received is actually supposed to be at the end of the file. For example open your file in a hex editor and type 'test123' at the end or something, and make sure it is actually sent and not received.

#7 Zomgbie   Members   -  Reputation: 236

Like
0Likes
Like

Posted 24 February 2011 - 08:55 AM

Tried that out now. All bytes are identical untill we hit the 508 mark, where i cannot compare anymore :(
Omg, zombie! Zomgbie.

#8 rip-off   Moderators   -  Reputation: 8049

Like
0Likes
Like

Posted 24 February 2011 - 09:22 AM

Does your server flush/close the socket after all bytes are sent?

#9 Zomgbie   Members   -  Reputation: 236

Like
0Likes
Like

Posted 24 February 2011 - 09:35 AM

Right now it does. It waits for 1 minut after sending, and then closes the socket. When i'm done with this, i'm not planning on closing the socket after the transfer. There is more data to be sent afterwards :)
Omg, zombie! Zomgbie.

#10 rip-off   Moderators   -  Reputation: 8049

Like
0Likes
Like

Posted 24 February 2011 - 09:46 AM

It should work. Try flushing the socket immediately after finishing anyway. You might also try disabling Nagle's algorithm, but I don't think that should be causing a problem as your clinet should be acknowledging every read().

Can you post a minimal client that demonstrates this behaviour? What about trying to readthe data byte by byte into the buffer? It does seem that your code has an issue when the "data left" drops below the size 16384.

#11 Drew_Benton   Crossbones+   -  Reputation: 1713

Like
0Likes
Like

Posted 24 February 2011 - 09:51 AM

Make sure you implement Erik's logic from post #4 for the send code as well on the server. Specifically, you need to make sure -1 is not returned on the send logic because you might be filling up the client's send buffer too fast on the server.

To help test that as well, you can increase the size of the send/recv buffers on sockets as well as disable nagle's algorithm. Check out the setsockopt function as well as the SO_RCVBUF, SO_SNDBUF, SO_LINGER, and TCP_NODELAY settings. You might need to Google for a few specific examples to read up on which settings might be best to try for your specific application.

Lastly, make sure you don't have any bugs in the actual debugging output statements you are using. I.e., I've done silly things in the past where I outputted a size before I modified it correctly so it always looked like there was data left, but there really wasn't. I'm not sure what your specific code looks like right now after modifications, but the last debug output seemed not to print enough variable information.

"But I, being poor, have only my dreams. I have spread my dreams under your feet; tread softly, because you tread on my dreams." - William Butler Yeats

#12 hplus0603   Moderators   -  Reputation: 5098

Like
-2Likes
Like

Posted 24 February 2011 - 01:26 PM

As the topic states, my recv() call keeps stopping when there is 508 bytes left to read. I'm trying to send a file, and it's all going good untill we hit the magical 508 bar.



People already talked about the while() loop on the sending side.
How do sender and receiver communicate the file size in the first place? If you print it before the while() loop on each side, do they agree?
enum Bool { True, False, FileNotFound };

#13 Zomgbie   Members   -  Reputation: 236

Like
0Likes
Like

Posted 25 February 2011 - 12:34 AM

The loop already checks for errors, and disconnects. But it never gets there, because recv() seems to wait for data thats not there.

Drew, i'll take a closer look at what you wrote when i get home.

Hplus, the client will send the file size and name as a structred packet. The server picks up on this and expect the next input on the wire to be the actual file. If i'm printing the file size on both server and client, theh are the same.

I should note that smaller files that only takes up a few loops, get through without problem.'i'll post the updated functions when i get home.

Thanks for all the awesome help so far :)
Omg, zombie! Zomgbie.

#14 Erik Rufelt   Crossbones+   -  Reputation: 3171

Like
0Likes
Like

Posted 25 February 2011 - 02:16 AM

You should get into the habit of posting your actual code, as the problem might very well be clearly visible there, and not pretend code that might not contain the same error. Include the parts that deal with sending the file size etc. before the file loop also.

#15 Zomgbie   Members   -  Reputation: 236

Like
0Likes
Like

Posted 25 February 2011 - 06:02 AM

You should get into the habit of posting your actual code...


Point taken! I've updated my first post with the two actual functions that takes care of sending/recv.

Edit: The formatting is a bit off, sorry for that.
Omg, zombie! Zomgbie.

#16 rip-off   Moderators   -  Reputation: 8049

Like
0Likes
Like

Posted 25 February 2011 - 06:24 AM

We still can't see your socket setup, and I don't see the server's socket being flushed or closed as previously suggested. If you could develop a minimal client (all in one file) that demonstrates the behaviour that would be easiest for us (as we can test it in isolation and see everything).

#17 Erik Rufelt   Crossbones+   -  Reputation: 3171

Like
0Likes
Like

Posted 25 February 2011 - 06:27 AM

How does recvCommand(client->clientSocket, NET_FILEINFO, buffer) work?
Make sure you don't read more than the number of bytes sent by the server's sendCommand(NET_FILEINFO, (char *)&info), or you might get part of the file in 'buffer' in that read. If you investigated the file-contents to be correct even when you don't receive the entire file that shouldn't be possible, but it seems the most likely issue from your code, so I would double-check that.

#18 hplus0603   Moderators   -  Reputation: 5098

Like
-1Likes
Like

Posted 25 February 2011 - 04:44 PM

Hplus, the client will send the file size and name as a structred packet. The server picks up on this and expect the next input on the wire to be the actual file. If i'm printing the file size on both server and client, theh are the same.


Are you sure that you're not accidentally reading more than was put in there for the file size packet?
Note that TCP is a stream protocol, not a packet protocol. You have to packetize and re-buffer yourself.
If you check the contents of the file that is received, are you missing the FIRST 508 bytes, or the LAST 508 bytes?
enum Bool { True, False, FileNotFound };




Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS