Lots of smaller packets, or one large packet?
Author
Hello,
I'm inheriting a large multiplayer racing game project.
I've been tasked with trying to optimize the already (seemingly) optimized networking code.
Currently, the system is server->client architected.
The games general update packet ( location, orientation, velocity, plus a few bits for game state info ) is packed down to ~35-37 bytes.
I've now learned via WireShark that my 35 byte packet "ballons" by an additional 42 bytes due to the ethernet frame data.
So, I'm "penalized" almost twice the size of my game data packet just for the ethernet header.
One of the seemingly "easy" action items involves the server, who handles addtional AI vehicles. Previously, when it was time for the server to update clients on his position, he'd automatically send out updates for each AI he was controlling as well. So, I'd have N packets ( N = server + number of AI ) sent, paying the ethernet frame size byte penalty for each of those N packets.
I've created a new "mega packet" for this case, where I bundle the server + all the AI data into one larger packet, and send that off. Pay the ethernet byte penalty just once.
As my data size checking surmised, my data load is almost exactly 1/2 as much.
Yah!
I think.
My tests are all between my two machines on my desktop ( on a LAN ), and not in the wilds of the internet.
My (first of many upcoming ) question is this: Is it considered "bad" to send smaller ( but larger ) packets contrasted to the numerous, smaller packets?
An obvious "bad" situation in my mind is if I "lose" a packet with the mega packet, I'll lose the update for all cars. ( Contrasted to the previous methodology, where losing a packet means I only lose data for a single car.)
Since you're limited by the ethernet protocol, you can carry a maximum of 1460 usable bytes per packet (maximum ethernet frame data: 1500 bytes, ip header: 20 bytes minimum, tcp header: 20 bytes minimum). If you would send much smaller packets, your ethernet, ip and tcp headers would contribute much more to the data you want to deliver, so you're quite right that sending bigger packets will improve performance.
I would consider it perfectly legal (and performance wise) to "batch" those packets, if it can be done. This is what the tcp socket on windows (I'm shure linux sockets do the same thing) for example does: It waits until a packet can be filled (all 1460 bytes) with data and then sends it to the receiving socket.
About packet loss: If your information is cruical that packet has to be sent again, no matter how big the packet was. I think the "best" size for a packet depends on the loss ratio, but in generel I'd say that you should try to fit as much data as possible into a single packet.
I would consider it perfectly legal (and performance wise) to "batch" those packets, if it can be done. This is what the tcp socket on windows (I'm shure linux sockets do the same thing) for example does: It waits until a packet can be filled (all 1460 bytes) with data and then sends it to the receiving socket.
About packet loss: If your information is cruical that packet has to be sent again, no matter how big the packet was. I think the "best" size for a packet depends on the loss ratio, but in generel I'd say that you should try to fit as much data as possible into a single packet.
Fewer, larger packets is more efficient - you only really need lots of small packets if you need interactivity with small amounts of data, characters typed into a remote terminal for instance. So I'd say collect the updates into larger packets.
Your game should be able to cope with the odd lost update for an individual vehicle so doing it for lots of them shouldn't cause too much of a problem.
One thing to watch out for is the maximum packet size that can be sent over your connection - the maximum transmission unit (or MTU). Over Ethernet, and most broadband connections, this is 1532 bytes before taking into account frame data. For dial up, it is typically 576 byte (if I remember correctly). You might want to limit your packets so they fit into the MTU rather than sending, say, a 40K packet - which will get split up by the IP stack anyway. Even if your local connection's MTU is 1532 bytes, that doesn't mean it won't get split up over the net if an intermediate connection can't handle such a large size. This is very rare though - most net infrastructure connections support ethernet sized packet because splitting them is relatively expensive.
Your game should be able to cope with the odd lost update for an individual vehicle so doing it for lots of them shouldn't cause too much of a problem.
One thing to watch out for is the maximum packet size that can be sent over your connection - the maximum transmission unit (or MTU). Over Ethernet, and most broadband connections, this is 1532 bytes before taking into account frame data. For dial up, it is typically 576 byte (if I remember correctly). You might want to limit your packets so they fit into the MTU rather than sending, say, a 40K packet - which will get split up by the IP stack anyway. Even if your local connection's MTU is 1532 bytes, that doesn't mean it won't get split up over the net if an intermediate connection can't handle such a large size. This is very rare though - most net infrastructure connections support ethernet sized packet because splitting them is relatively expensive.
Author
Wow, great responses.
And thanks (both) for pointing out MTU.
It's doubtful we'll hit our 30 player "ideal", but that would put us at around 1200 bytes ( 30 players * ~40 bytes per player )... so we'd still fit under the MTU.
If for some reason we'll by-pass the MTU, then breaking into multiple "mega" packets will still be a win.
And thanks (both) for pointing out MTU.
It's doubtful we'll hit our 30 player "ideal", but that would put us at around 1200 bytes ( 30 players * ~40 bytes per player )... so we'd still fit under the MTU.
If for some reason we'll by-pass the MTU, then breaking into multiple "mega" packets will still be a win.
Actually, the Ethernet overhead doesn't matter, because it's stripped away when it goes to your internet gateway. Physical layer framing, in general, isn't counted when you count networking costs, because it both varies, and you can do nothing about it. (For example, on ATM, each ATM cell takes 53 bytes on the wire, of which 48 bytes is data for your packets).
What does add space is the IP and TCP headers. The IP header is 20 bytes. The TCP header starts out at 20 bytes, but may add additional size based on options used. 42 bytes seems weird, though, because TCP headers are generally aligned to 4 bytes, so you'd either get 40 bytes or 44 bytes of header overhead.
Note that the overhead for UDP is only 8 bytes (so 28 bytes total with UDP/IP), but you have to do a lot of things yourself that TCP does for you (like congestion control, windowing, re-sends etc).
That said, it's always a good idea to bake all your messaging for the current tick into a single packet, and send that single packet. That saves on both header overhead and packet framing (your own sequence numbers, checksums, security keys etc can all be sent once per packet, instead of once per message).
Edit: ETHERNET overhead is stripped away. TCP overhead matters.
[Edited by - hplus0603 on January 21, 2009 12:28:26 PM]
What does add space is the IP and TCP headers. The IP header is 20 bytes. The TCP header starts out at 20 bytes, but may add additional size based on options used. 42 bytes seems weird, though, because TCP headers are generally aligned to 4 bytes, so you'd either get 40 bytes or 44 bytes of header overhead.
Note that the overhead for UDP is only 8 bytes (so 28 bytes total with UDP/IP), but you have to do a lot of things yourself that TCP does for you (like congestion control, windowing, re-sends etc).
That said, it's always a good idea to bake all your messaging for the current tick into a single packet, and send that single packet. That saves on both header overhead and packet framing (your own sequence numbers, checksums, security keys etc can all be sent once per packet, instead of once per message).
Edit: ETHERNET overhead is stripped away. TCP overhead matters.
[Edited by - hplus0603 on January 21, 2009 12:28:26 PM]
The only times you can't efficiently combine multiple packets into one mega-packet are when:
-you're sending the smaller packets to different targets
-you're sending the smaller packets at different times
In all other cases (i.e. you're sending multiple packets to the same target at the same time), they can be safely combined and as long as they stay under MTU, you'll probably win.
Feel free to correct me if I said something out of place.
-you're sending the smaller packets to different targets
-you're sending the smaller packets at different times
In all other cases (i.e. you're sending multiple packets to the same target at the same time), they can be safely combined and as long as they stay under MTU, you'll probably win.
Feel free to correct me if I said something out of place.
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
