Archived

This topic is now archived and is closed to further replies.

MMOG packet sending rate

This topic is 5015 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Packets for EQ range anyway from 10 bytes (basic movement packet) on up to 500 bytes for stuff requreing more information (doing a list of peope in the current zone). Some structres are greater than 500 bytes, but EQ''s network layer splits these up into 500 byte packets and ships them out separately.


Watching the traffic across the network it seems that on the low end EQ sends about 1-5 packets per second. Probablly 15-20 in higher traffic situations(I''ll have to check later) and prolly combines some packets before sending em, bumping up the byte count per packet.

-=[ Megahertz ]=-

Share this post


Link to post
Share on other sites
Here''s a tcpdump during a raid in EQ. This is with about 50 other players in very close proximity.

The number after udp is the size of the packet including the udp header(28 bytes i believe)

lines with eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx are packets sent to me

and

xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com

are packets sent to the server.



tcpdump: listening on eth0
17:37:30.176150 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 58 [tos 0x20]
17:37:30.258102 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 219 [tos 0x20]
17:37:30.264923 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 24
17:37:30.346763 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 10 [tos 0x20]
17:37:30.488608 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 114 [tos 0x20]
17:37:30.538719 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:30.567065 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 62 [tos 0x20]
17:37:30.723540 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 130 [tos 0x20]
17:37:30.773615 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:30.958266 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 84 [tos 0x20]
17:37:30.958463 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 50 [tos 0x20]
17:37:31.008246 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:31.191557 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 89 [tos 0x20]
17:37:31.241992 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:31.348423 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 101 [tos 0x20]
17:37:31.428108 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 79 [tos 0x20]
17:37:31.477864 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:31.660640 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 82 [tos 0x20]
17:37:31.739981 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 133 [tos 0x20]
17:37:31.789919 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:31.975606 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 132 [tos 0x20]
17:37:31.975790 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 70 [tos 0x20]
17:37:32.025709 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:32.207440 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 114 [tos 0x20]
17:37:32.257364 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:32.285372 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 58 [tos 0x20]
17:37:32.442622 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 122 [tos 0x20]
17:37:32.492194 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:32.676751 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 123 [tos 0x20]
17:37:32.677367 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 106 [tos 0x20]
17:37:32.726950 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:32.989179 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 62 [tos 0x20]
17:37:33.067011 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 105 [tos 0x20]
17:37:33.117314 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:33.304258 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 94 [tos 0x20]
17:37:33.304444 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 58 [tos 0x20]
17:37:33.354049 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:33.536459 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 114 [tos 0x20]
17:37:33.586735 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:33.613901 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 86 [tos 0x20]
17:37:33.926494 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 116 [tos 0x20]
17:37:33.977107 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:34.083528 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 50 [tos 0x20]
17:37:34.169685 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 90 [tos 0x20]
17:37:34.220876 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:34.394760 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 32 [tos 0x20]
17:37:34.444846 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:34.472687 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 77 [tos 0x20]
17:37:34.865747 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 104 [tos 0x20]
17:37:34.866018 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 131 [tos 0x20]
17:37:34.916122 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:34.957834 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 22
17:37:35.008785 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 41
17:37:35.035135 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 10 [tos 0x20]
17:37:35.184557 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 70 [tos 0x20]
17:37:35.256930 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 106 [tos 0x20]
17:37:35.307377 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:35.454001 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 41
17:37:35.490555 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 113 [tos 0x20]
17:37:35.568383 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 101 [tos 0x20]
17:37:35.618486 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:35.764463 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 41
17:37:35.803761 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 102 [tos 0x20]
17:37:35.804005 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 101 [tos 0x20]
17:37:35.853259 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:36.114311 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 41
17:37:36.201213 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 214 [tos 0x20]
17:37:36.201400 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 125 [tos 0x20]
17:37:36.251518 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:36.427135 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 95 [tos 0x20]
17:37:36.433240 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 24
17:37:36.472372 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 41
17:37:36.504482 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 52 [tos 0x20]
17:37:36.661075 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 38 [tos 0x20]
17:37:36.711007 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:36.818614 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 125 [tos 0x20]
17:37:36.974521 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 104 [tos 0x20]
17:37:37.025118 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:37.286717 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 119 [tos 0x20]
17:37:37.308060 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 41
17:37:37.364158 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 80 [tos 0x20]
17:37:37.415441 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:37.601101 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 92 [tos 0x20]
17:37:37.633394 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 43
17:37:37.677736 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 107 [tos 0x20]
17:37:37.976589 eqchannel1.station.sony.com.sd > xxx.xxx.xxx.xxx.3330: udp 56 [tos 0x20]
17:37:37.989384 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 129 [tos 0x20]
17:37:37.989741 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 77 [tos 0x20]
17:37:37.993863 xxx.xxx.xxx.xxx.3330 > eqchannel1.station.sony.com.sd: udp 6
17:37:38.039514 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:38.311337 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 89 [tos 0x20]
17:37:38.444664 eqchannel1.station.sony.com.sd > xxx.xxx.xxx.xxx.3330: udp 65 [tos 0x20]
17:37:38.482959 xxx.xxx.xxx.xxx.3330 > eqchannel1.station.sony.com.sd: udp 6
17:37:38.531488 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 41
17:37:38.614633 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 143 [tos 0x20]
17:37:38.849663 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 104 [tos 0x20]
17:37:38.899241 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:38.927765 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 107 [tos 0x20]
17:37:39.105579 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 41
17:37:39.163665 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 160 [tos 0x20]
17:37:39.213495 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:39.326412 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 89 [tos 0x20]
17:37:39.379593 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 41
17:37:39.397231 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 133 [tos 0x20]
17:37:39.447315 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:39.692827 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 41
17:37:39.786564 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 89 [tos 0x20]
17:37:39.786751 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 101 [tos 0x20]
17:37:39.836346 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:40.012293 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 41
17:37:40.020781 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 123 [tos 0x20]
17:37:40.071401 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:40.177298 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 70 [tos 0x20]
17:37:40.346028 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 41
17:37:40.489774 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 82 [tos 0x20]
17:37:40.567933 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 82 [tos 0x20]
17:37:40.618023 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:40.654744 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 41
17:37:40.802257 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 94 [tos 0x20]
17:37:41.036257 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 32 [tos 0x20]
17:37:41.059825 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 43
17:37:41.192428 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 62 [tos 0x20]
17:37:41.375177 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 41
17:37:41.505216 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 95 [tos 0x20]
17:37:41.584225 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 101 [tos 0x20]
17:37:41.635522 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:41.818030 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 125 [tos 0x20]
17:37:41.902509 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 41
17:37:42.131575 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 95 [tos 0x20]
17:37:42.211972 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 253 [tos 0x20]
17:37:42.227161 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 43
17:37:42.483200 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 41
17:37:42.523514 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 125 [tos 0x20]
17:37:42.727259 eqchannel1.station.sony.com.sd > xxx.xxx.xxx.xxx.3330: udp 55 [tos 0x20]
17:37:42.761417 xxx.xxx.xxx.xxx.3330 > eqchannel1.station.sony.com.sd: udp 6
17:37:42.761603 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 41
17:37:42.834302 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 101 [tos 0x20]
17:37:43.062442 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 41
17:37:43.070935 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 22 [tos 0x20]
17:37:43.121201 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:43.146294 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 89 [tos 0x20]
17:37:43.390691 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 90 [tos 0x20]
17:37:43.441289 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:43.537106 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 149 [tos 0x20]
17:37:43.771599 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 126 [tos 0x20]
17:37:43.821522 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:43.849862 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 113 [tos 0x20]
17:37:44.005771 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 57 [tos 0x20]
17:37:44.055352 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 10
17:37:44.162725 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 94 [tos 0x20]
17:37:44.348123 xxx.xxx.xxx.xxx.3428 > eqdynz-028.989studios.com.1413: udp 41
17:37:44.475766 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 111 [tos 0x20]
17:37:44.475954 eqdynz-028.989studios.com.1413 > xxx.xxx.xxx.xxx.3428: udp 107 [tos 0x20]


-=[ Megahertz ]=-

Share this post


Link to post
Share on other sites
to add to MHz description of the above dump, the packets to/from eqchannel1 are chat messages made up entirely of plain text. Only a small portion of chat goes to the channel server, but nonetheless it will help to explain some of the odd packet sizes/range.

quote:
prolly combines some packets before sending em, bumping up the byte count per packet.


Yep. writev() is a wonderful thing...

Share this post


Link to post
Share on other sites
You don''t actually need writev() to combine packets. When you''re using game data rates (less than 5 kB per second) them assembling the full packets using memcpy() has no noticeable impact on frame rate and CPU utilization, compared to writev() or similar scatter/gather mechanisms.

Share this post


Link to post
Share on other sites
I have measured this when playing Runescape and was surprised that it's about only 300-500 bytes/second. And that was with dozens of players walking around and mining. Not a lot at all, and if you stand still and are in a quiet area it drops to almost nothing.
RPG's don't need the 20 updates/second that shooters need because combat is different, you don't aim guns at others, so only one update/second of NPC positions is enough if the clients walks the NPC's around smoothly and occasionally corrects them.

[edited by - Fidelio66 on March 23, 2004 12:08:25 PM]

Share this post


Link to post
Share on other sites
quote:
Original post by hplus0603
You don't actually need writev() to combine packets. When you're using game data rates (less than 5 kB per second) them assembling the full packets using memcpy() has no noticeable impact on frame rate and CPU utilization, compared to writev() or similar scatter/gather mechanisms.


Please keep in mind that we're talking about massively multiplayer games here. You might only send 1-2KB/sec, but you are potentially sending to many players. There is no harm in being more efficient where possible. With writev you avoid function call overhead and kernel memory locking that you get by making multiple send() calls. The memcpy()s get expensive, and so do the memory allocations to store the combined buffer in application space. In one of the network libraries I worked on at SOE, we achieved an average 10X throughput increase by reducing memcpy()s in a single layer of code. It will make a huge difference on the server, less so on clients (or between peers). Oh, and the reason I mention writev here is because it's used in EQ.

[edited by - fingh on March 23, 2004 1:41:41 PM]

[edited by - fingh on March 23, 2004 1:50:47 PM]

Share this post


Link to post
Share on other sites
Calling send() more often would indeed be dumb. And who would use system allocation to create a buffer? That'd be dumb, too, when you have stack, or statics, or pre-allocated buffers (depending on your technique) at your disposal.

I suppose whether memcpy() will be slow or not depends entirely on your server structure, and whether it's useful to already have the data in cache. The cost of the memcpy() is the cache line reads for the sources, and the cache line writes for the destination buffer. However, then the data is already in the cache. If you writev(), the data will likely need to be coalesced by the driver anyway, and the data is then NOT in cache, so the cost really is quite minimal in many situations.

Many network cards do not have extensive scatter/gather controllers; typically they just have a single, contiguous ring buffer, so some assembling has to be done by the CPU somewhere -- in the app, or in the driver. In the ideal I/O architecture, you get handed buffers by the driver layer that you can put data in, to avoid copying. Alas, neither UNIX nor Windows usually expose this model (except for some DirectX cases). If you're working on custom hardware or OS stuff, though, you have more liberty to put work where it should go :-)

Depending on size of packets, and fragmentation, you may actually touch MORE cache lines assembling your writev() array, and then having the driver re-assemble it, than just assembling it yourself. In all honesty, you'd need to be REALLY fragmented for that to be the case, though.

Edit: actually, let's do the math: A maximum of 5 kilobytes per player, per second. 1,000 players per zone (I think EQ only went to 150 before it started lagging like mad). That's a whopping 5 megabytes per second of data to coalesce -- out of a typical current server memory bandwidth of 5 gigabytes per second.

I agree that removing memcpy() from places where it doesn't belong is beneficial. However, assembling packets for final delivery to the driver is seldom one of those places -- I note that you don't specify which particular subsystem, or how much data it was, or whether the 10x improvement actually impacted the overall profile at all :-)


[edited by - hplus0603 on March 24, 2004 1:37:15 AM]

Share this post


Link to post
Share on other sites
Fidelio66 brings up an interesting question here. How much packet rate is enough for MMORPG? We certainly neither can require nor afford 20 per-second, but is 1 per second is enough for a good playing experience? I think the character movement is the most demanding part for packet rate and 5 to 10 updates per-second is reasonable, like that in EQ. But I''d also like to make it drop down to 1 per-second if possible.

Another question is, how much performance can we gain to combine several small packets into a big one? The traffic saving should come from reducing the packets header overhead. The drawback is, more packets are easier to lose. Is there a issue that more packets increase the server routing burden through the network?

Certainly higher update rate is better for the game play so if it won''t make too much sense to reduce that, why not do it?

Share this post


Link to post
Share on other sites
If players and NPC''s walk around, because you are not pointing guns and shooting, it''s not neccesary that the client renders them in the position they are at on the server. It does not need to be exact, nobody will notice if an NPC that walks is at position x at time t while on the server he''s already 2 steps further, or even when clients put him in a slightly different position. When combat starts you can give priority to fighting characters. Apart from that, you can make the client a bit smarter and make the characters walk around sufficiently correct based on occasional position updates.

Share this post


Link to post
Share on other sites
I believe the MMORPG programming book (forget the name) suggests keeping a priority queue of entities per player, sorted on when to send the next update. Each time it''s time to update the player, elements are pulled off the queue, one update packet is generated (for all of the elements pulled off), and the elements are re-inserted into the queue, at time (now + f(d)) where f(d) is a function of distance from player to entity. That way, closer entities get more frequent updates.

There are a few problems with this, such as using binoculars, or when you''re moving towards an entity. If the entity was far away, but you''re moving towards each other, then you might want an update sooner than the position in the queue would indicate. I believe the solution recommended in that book was to inspect the queue every so often (every 5 seconds?) and fix up anything that looks out of order.

You absolutely want to coalesce updates. If you don''t, you''re wasting at least half your bandwidth. A typical entity update packet has no more than 20 bytes per entity; a typical UDP header (with IP header and ATM framing) is bigger than that. If you drop a packet, all the entities in the packet will be lost, true, but Internet packet dropping is not random; it''s bursty. Thus, several small packets sent right after each other are likely to all get dropped together, anyway. You might as well save the header overhead, and support more simultaneous items in your world.

We send 10 packets per second (plus chat, transactions, etc).

Share this post


Link to post
Share on other sites