Sign in to follow this  
uri8700

TCP question

Recommended Posts

uri8700    100
Hi all

If I send 60,000 data packets of 20 bytes each using send() and TCP, how much header overhead am I gonna get? I mean, I hope not every send() warrants its own TCP packet with a full header, does TCP aggregate send()s and waits for data to pile up before forming a packet? How long does it wait before sending a packet if no more data is sent()?

Thanks

Share this post


Link to post
Share on other sites
ApochPiQ    22999
Have a look into the "Nagle algorithm" and how it is commonly implemented by operating systems; that should be illuminating [img]http://public.gamedev.net/public/style_emoticons/default/smile.gif[/img]

Share this post


Link to post
Share on other sites
uri8700    100
Thanks, you saved me tons of trouble by pointing me in this direction.

Now I see that it *is* necessary to send each data packet in its own TCP packet, otherwise timing gets thrown off.

So for a real-time game with movement orders and object positions being sent in a flux of 20-byte packets, you'd have to set the TCP_NODELAY option, to disable nagle's. Is it also necessary to set SO_SNDBUF to 0 and disable Winsock buffering? it seems so, otherwise you still get disrupted timing from buffering.

Share this post


Link to post
Share on other sites
SiCrane    11839
Keep in mind that even if you disable Nagle, there's no guarantee that every send() call will be a single packet.

Share this post


Link to post
Share on other sites
frob    44903
[quote name='_TL_' timestamp='1311125158' post='4837736']
Now I see that it *is* necessary to send each data packet in its own TCP packet, otherwise timing gets thrown off.

So for a real-time game with movement orders and object positions being sent in a flux of 20-byte packets, you'd have to set the TCP_NODELAY option, to disable nagle's. Is it also necessary to set SO_SNDBUF to 0 and disable Winsock buffering? it seems so, otherwise you still get disrupted timing from buffering.
[/quote]

Relying on network packets for timing is a bad idea. Basically, it won't work.

Any traffic on the network can throw off the timings horribly. Any network congestion, network rerouting, dropped packets, electrical storms with lightning in the air, solar flares, or just about anything else can throw it off. These are entirely outside your control.

For example, you may be running with almost exactly 30ms latency for the first thousand packets, but then the next one has 47ms latency, another has 3124ms latency, then 54, then drift to a new stable timing of 37ms latency.

This is true regardless of your choice of using TCP, UDP, or other protocols.


The time it takes to travel the network is entirely outside the scope of socket programming. All you know is that it will probably get there eventually. It may travel by fiber, satellite around the world multiple times, copper cable around a room, or it may travel by [url="http://www.blug.linux.no/rfc1149/writeup.html"]messages strapped to pigeons[/url], wireless [url="http://www.turnpoint.net/wireless/has.html"]Pringles-can antenna[/url], or even [url="http://eagle.auc.ca/~dreid/"]bongo drums[/url]. That takes place at a network layer far below those you control.


Do not make assumptions about how long it takes, because it is highly variable and completely outside your control.

Share this post


Link to post
Share on other sites
uri8700    100
@frob: LOL maybe I'll try the bongo drums method to improve latency

But seriously now, I'm having big problems with latency. When I run it on loopback it's perfect, but when I try it on a 170ms latency broadband WAN, I get tons of trouble. Most obviously, animation becomes jumpy and totally not smooth. Why is it? is it because I'm using TCP? I would have thought high latency only causes a delayed response time for the first packet, but then it's supposed to be smooth cause there's a stream of packets keeping it going. But it seems as if the client waits the 170ms for every packet seperately.

Share this post


Link to post
Share on other sites
hplus0603    11347
[quote name='_TL_' timestamp='1311292948' post='4838710']
obviously, animation becomes jumpy and totally not smooth. Why is it? is it because I'm using TCP? I would have thought high latency only causes a delayed response time for the first packet, but then it's supposed to be smooth cause there's a stream of packets keeping it going. But it seems as if the client waits the 170ms for every packet seperately.
[/quote]

You probably aren't properly decoupling your simulation, your networking, and your rendering.


Generally, when you receive a network update, you should use that as information about what you want the simulation to be in the future -- how to go from where it's at, to some guess at what it will be in the future. Future network packets keep updating what your predicted future state will be, so you keep striving at that state. This is typically based on step numbers, where each step length is the same fixed value.

Your rendering typically interpolates between the previous physics frame and the next physics frame based on the current wall-clock time and an estimate of when the next physics step will be taken in wall-clock time. Or it just renders what the last physics step outcome was; that's simpler, but less smooth in some cases.

Share this post


Link to post
Share on other sites
uri8700    100
My networking & rendering are tightly knit, via the data object that keeps object position, gets updated by the network and gets redrawn much quicker than network-updated. What paces the whole thing is the rate at which the network updates the object, and somehow latency slows that down to ~5-6 times per second, instead of 30 or so.

Share this post


Link to post
Share on other sites
hplus0603    11347
The object shouldn't "jump" when it gets a new network state. Instead, it should just use the new state as another data point in an algorithm that estimates where the objects should be at each point in time. Some people use this by interpolating between the position the object was last in and the position you received; others do it by forward extrapolating from the received position/velocity; some do it by interpolating between two forward extrapolated positions from the last and current packets; there are many ways to do it.

Halo runs at 10 or 15 updates per second, but renders at 60 frames per second, and manages pretty smooth game play, for example.

If you get 5-6 packets per second, have you looked at the data in a network sniffer (like WireShark) to figure out what's actually going on the wire? Are you getting "clumping" of packets, many in a single packet? If so, you're probably seeing the effects of Nagle's algorithm, which you can turn off using TCP_NODELAY.

Share this post


Link to post
Share on other sites
You may want to try Ryzom (free to play trial, plus the source code is available) just to see live -- and at the same time -- how it this is done [i]properly [/i]and how you should [i]not at all[/i] do it.

For an example of how it's done right, watch avatars the first as the game just finished loading. You'll see that they are running rather than walking (and, with proper running animations) to catch up with the updated positions that were delivered while the game blocked on I/O. Play on a bad wireless home network or start a webserver stress test tool while playing, and look how the game behaviour changes. The game really does its best at making it look kind of natural or at least believeable.

At the other hand, run side-by-side with someone else. You will inevitably notice that one of you runs faster. Why? Because how fast you run depends on your network latency. In a game which includes PvP, having the speed at which you can run depend on your network speed is a funny feature.

Now load a second client and watch your first character while running a stresstester or something similar. Or, watch someone else who has a terribly noisy line with packet loss in the range of 60-80%. You'll see that they "teleport" 2-3 meters at a time instead of moving.

Teleport to a busy place with [i]lots of [/i]individual (and moving) entities (any major city on the mainland) and try to rotate the view with the mouse. You will wonder why it does not work. That is because rendering and networking run in lockstep. During the 1-2 seconds while a huge bulk of data is coming over the wire, the screen is not redrawn, and no messages are processed. You cannot move, you cannot even change the view. If you were unlucky enough to hold down both mouse buttons during that time, it may be that the game registered that and set your character into running mode. Now, if you drop the mouse during that time because you see that the screen is frozen, chances are that the "mouse up" event is swallowed (no idea how [i]that [/i]happens), and your character will be running in some random direction with the view rotated to a random angle when it resumes. Fun. Luckily you can't drown or fall off cliffs.

Go to a place with an [i]extreme [/i]amount of simular, moving entities (a kitin hole). You will see that even on a very high end machine, you get something around 5-6 fps (less if you don't have broadband). That doesn't seem make any sense at all, especially since GPU performance counters show that there is hardly any load, and the CPU is only loaded 80-90% on a single core, too.
The closer you get, the lower the FPS. That is, again, because rendering and networking go lockstep, and the closer you get, the more entity updates are sent between frames (luckily, not all in one bulk, or you couldn't move at all in between). The game uses a quite sophisticated algorithm to optimize updates which is described at Gamasutra, but it doesn't help in the end in such a situation. The rendering-coupled-to-networking implementation just kills it.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this