-SOLVED- TCP Message data not all coming through
Author
I have a client and server program running. Most of my messages go through pretty easily, even when they are a couple hundred bytes. Eventually the client gets to a point where it can request the latest version of a map, and the server will send it to them.
When I put a break point in the recv() call of my code, I get the entire 6783 bytes. If I put the break point a bit farther down the line instead, or have no break point at all, I'll only have gotten 6781 bytes. This is consistent... and I have no idea why.
Can anyone shed some light on this? Is it perhaps a normal issue that I just need to tweak some values for?
[Edited by - ShinkaFudan on February 3, 2010 11:31:44 PM]
TCP is a stream. You could get any number of bytes from the recv call. If you send 100 bytes there's nothing to say you can't get 10 bytes after calling recv. For a packet of your size this is more common. So call recv and keep building your packet with multiple recv calls.
Author
Quote:Original post by Sirisian
TCP is a stream. You could get any number of bytes from the recv call. If you send 100 bytes there's nothing to say you can't get 10 bytes after calling recv. For a packet of your size this is more common. So call recv and keep building your packet with multiple recv calls.
Sorry. I should have emphasized that I already do this. The message sometimes will come in parts, a couple thousand bytes each. When it doesn't work, it's because the number of bytes that it all adds up to is 2 bytes short.
Also after further testing, I've discovered it is not at all consistent as I thought it was. About half the time I get all of the data. The other half I am 2 bytes short.
"the number of bytes that it all adds up to is 2 bytes short."
Is there any chance this is due to some sort of pointer alignment problem? {eg; you're reading into an array of ints, but the message can be truncated on byte boundaries}
Can you write a small test case which sends test patterns through. Then you can send "0123456789" repeats and see whether the bytes are missing from the start/middle/end by checking the results?
If you put a "sleep()" call in your program about where you put the breakpoint, do you also get the full data? If so, this is a timing related thing.
How do you know the data is coming up short -- is it possible it's the check that has the wrong logic rather than the data actually being short?
Does it *always* fail without the pause? Network problems (dropped packets etc) are random(ish) -- if the transmission always fails the same way, then there's logic wrong somewhere.
Is there any chance this is due to some sort of pointer alignment problem? {eg; you're reading into an array of ints, but the message can be truncated on byte boundaries}
Can you write a small test case which sends test patterns through. Then you can send "0123456789" repeats and see whether the bytes are missing from the start/middle/end by checking the results?
If you put a "sleep()" call in your program about where you put the breakpoint, do you also get the full data? If so, this is a timing related thing.
How do you know the data is coming up short -- is it possible it's the check that has the wrong logic rather than the data actually being short?
Does it *always* fail without the pause? Network problems (dropped packets etc) are random(ish) -- if the transmission always fails the same way, then there's logic wrong somewhere.
Author
Quote:Original post by Katie
"the number of bytes that it all adds up to is 2 bytes short."
Is there any chance this is due to some sort of pointer alignment problem? {eg; you're reading into an array of ints, but the message can be truncated on byte boundaries}
Can you write a small test case which sends test patterns through. Then you can send "0123456789" repeats and see whether the bytes are missing from the start/middle/end by checking the results?
If you put a "sleep()" call in your program about where you put the breakpoint, do you also get the full data? If so, this is a timing related thing.
How do you know the data is coming up short -- is it possible it's the check that has the wrong logic rather than the data actually being short?
Does it *always* fail without the pause? Network problems (dropped packets etc) are random(ish) -- if the transmission always fails the same way, then there's logic wrong somewhere.
Since I'm sending 6783 and getting 6781, I don't believe this could be a pointer alignment problem. This is particularly true of ints, since 6780 and 6784 are divisible by 4, and the numbers are between those two. Perhaps if one of the numbers were divisible by 4... but I'm taking the data into a char buffer, using the standard recv() call, so even if it were a possibility with the number of bytes, I don't think that is the problem.
I think I've found the problem though. I have a TCP wrapper that uses a value called "format" that can be 0, 1, or 2. If it is 0 (which in my program, it is), there is an unsigned short attached to the message and detached on the other end for size checking reasons. I think that since I'm receiving the message in two parts but only sending it once, it may be taking 2 bytes out both times and therefore I'm short 2 bytes.
Sadly, I didn't realize this was the problem until just now, and I don't get home for another 8 hours to test my theory. I'll report back when I find out.
[Edited by - ShinkaFudan on February 3, 2010 11:11:35 AM]
It appears you are using different packing instructions.
In c++ a structure without specifying a packing algorithms has packing header wich sometimes is 4 to 16 bites.
See this example:
struc myStructure
{
int i;
int a;
};
sizeof(myStructure) is not 8 bits, it could be 12 16 or even 24 depending the compiler!
In c++ a structure without specifying a packing algorithms has packing header wich sometimes is 4 to 16 bites.
See this example:
struc myStructure
{
int i;
int a;
};
sizeof(myStructure) is not 8 bits, it could be 12 16 or even 24 depending the compiler!
Author
Quote:Original post by becoolnike
It appears you are using different packing instructions.
In c++ a structure without specifying a packing algorithms has packing header wich sometimes is 4 to 16 bites.
See this example:
struc myStructure
{
int i;
int a;
};
sizeof(myStructure) is not 8 bits, it could be 12 16 or even 24 depending the compiler!
I should have been more specific. This could not be the problem, because I am just putting pure data types into the stream. char, int, and float only. Also, I am using the same compiler and OS for both the server and client, so the sizes of the data are the same. Thank you for this though. This reminder about structure padding may prevent a problem later in my project.
Author
Alright, I went home and tested my theory and it was correct. I wouldn't have thought of it without all of your help, so thank you very much.
For future notice:
- If a TCP packet gets an int put at the front of the buffer before being sent off, to be read when a piece of data is received, then make sure your system doesn't allow the reading to happen twice in a way that pulls the data out, as the second piece of the split packet won't have the int tied to it, and you'll end up trashing your data.
For future notice:
- If a TCP packet gets an int put at the front of the buffer before being sent off, to be read when a piece of data is received, then make sure your system doesn't allow the reading to happen twice in a way that pulls the data out, as the second piece of the split packet won't have the int tied to it, and you'll end up trashing your data.
Would a simpler way to read your note be this?
"Each call to recv() or read() will remove data from the head of the stream, leaving less data for further reads."
"Each call to recv() or read() will remove data from the head of the stream, leaving less data for further reads."
Author
Quote:Original post by hplus0603
Would a simpler way to read your note be this?
"Each call to recv() or read() will remove data from the head of the stream, leaving less data for further reads."
Yes, but the main point I was trying to make is that you can't assume a packet won't be split in two, so if you have a system that calls recv() on each part coming in, it will take data from the second half of a split message and screw up your data, especially if you are taking an int from the stream that you use to create a buffer based on the size.... you would grab junk data and possibly create gigs of space in a buffer.
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement