what is the most effective way to detect broken connection?
consider MMORPG. lobby (central) server keeps all client connections mainly to prevent
logging in the same account. it should support thousands of clients. to support 1 min connection timeout it should send keep-alive packets every 30 secs to each client (cos almost all connections are idle). so having let's say 5K clients lobby will need to send keep alive packet every 6ms. sounds like a bottleneck. do i miss anything?
is it more efficient to use TCP SO_KEEPALIVE instead of hand-made keep-alive packets? if I'll configure TCP keep alive options properly (let's say TCP_KEEPCNT = 1, TCP_KEEPINTVL = [any] and TCP_KEEPIDLE = 30), should I configure also any timeout to detect disconnection on 1 min timeout?
effective keep-alive?
I'm no network programmer, but I've dealt with a similar issue with business software that I work on at work.
5,000 clients on a single server seems like quite a few. Basically you want each client to send a keepalive at a specific interval (for our purposes, I think ours is ever two minutes or so). If the server misses getting a keepalive from a client, it assumes it is dead and trims its connection. Mind you, at work I deal with a lot fewer connections (I think ~200 simultaneous connections is what I've been told). From what I remember of working with keepalives, it was the client that sends them out to the server, then on a periodic basis the server cleans out any dead connections.
I guess it really depends on what you are doing, but you could scale up the amount of time required for someone to time out, do some testing, and narrow it down if necessary. Just make things configurable on both sides and play around with it a bit.
Mind you, take everything I've said with a grain of salt. I haven't worked with keepalives in a game development setting, only business software so things may be quite different.
5,000 clients on a single server seems like quite a few. Basically you want each client to send a keepalive at a specific interval (for our purposes, I think ours is ever two minutes or so). If the server misses getting a keepalive from a client, it assumes it is dead and trims its connection. Mind you, at work I deal with a lot fewer connections (I think ~200 simultaneous connections is what I've been told). From what I remember of working with keepalives, it was the client that sends them out to the server, then on a periodic basis the server cleans out any dead connections.
I guess it really depends on what you are doing, but you could scale up the amount of time required for someone to time out, do some testing, and narrow it down if necessary. Just make things configurable on both sides and play around with it a bit.
Mind you, take everything I've said with a grain of salt. I haven't worked with keepalives in a game development setting, only business software so things may be quite different.
Quote:Original post by @
so having let's say 5K clients lobby will need to send keep alive packet every 6ms. sounds like a bottleneck. do i miss anything?
- If client did something, sent something, that counts as keepalive, just reset the counter
- worst-case 2 keepalives per minute = 10000/60 = 166 packets per second - that is not much, peak might be measured in tens of thousands of packets per second
- TCP's built-in keepalive might not be needed, depending on usage patterns. If there is at least minimal activity from each client, using that as indicator is enough
Also, when talking about concurrent numbers, I think the now long gone lugdunum server is right up there:
Quote:New in version 16.45 (March 31th 2004) :
* Razorback server (dual Opteron, 6 GB memory) handled 600.000 clients with this version.
Quote: New in version 17.1 (December 1st 2004) :
* Razorback2 server can host more than 900.000 clients.
And the server did keepalives, matchmaking and search. All on single machine.
Unless you’re streaming video, audio or something else just use TCP. It’s secure, reliable, performs better and connection based so you don’t have to deal with keep-alive. Not to mention most of the hardware out there today is optimized for TCP traffic (hence the better performance, although the fact that ISPs tend to deprioritize UDP traffic plays a big role as well).
Quote:Is it more efficient to use TCP SO_KEEPALIVE instead of hand-made keep-alive packets?TCP keepalive messages carry no payload, so yes, they're technically more efficient, but the difference is negligible. More importantly, they free you from having to worry about it.
Application-level keepalives have other nice properties, such as allowing for time synchronization and whatnot. I'd probably send a keepalive every 10 seconds, because I want faster disconnects when there is no response. If there's a 30 second interval without anything from the client, and the client sends keepalives every 10 seconds, then that's a good reason to close the connection.
Also, note that you don't need keep-alives both directions. Because the server will be sending a lot more data than receiving, you want the keep-alive to go the other way. The client doesn't really need a keep-alive from the server anyway (necessarily).
If they're all hitting a database all the time, sure!
If all they do is chat and occasional matchmaking, then, as Antheus says, you should be able to do 100,000 on a normal server machine with no sweat.It's more a question of what quality server bandwidth and hosting you want to pay for -- as in, how many players can you really afford? Technology is unlikely to be the limiting factor, unless you're at "real bussiness scale."
Also, note that you don't need keep-alives both directions. Because the server will be sending a lot more data than receiving, you want the keep-alive to go the other way. The client doesn't really need a keep-alive from the server anyway (necessarily).
Quote:5,000 clients on a single server seems like quite a few.
If they're all hitting a database all the time, sure!
If all they do is chat and occasional matchmaking, then, as Antheus says, you should be able to do 100,000 on a normal server machine with no sweat.It's more a question of what quality server bandwidth and hosting you want to pay for -- as in, how many players can you really afford? Technology is unlikely to be the limiting factor, unless you're at "real bussiness scale."
Author
is it possible by boost ASIO to receive empty payload TCP packet (keep-alive packet), just to update timestamp of the last client activity?
tried:
- boost::asio::async_read(socket, boost::asio::buffer((void*)0, 0), handler) - causes "end of file" immediately
- boost::asio::async_read(socket, boost::asio::buffer((void*)0, 0), boost::asio::transfer_at_least(0), handler) - no "end of file" but handler is triggered immediately before any data was actually sent
trying to avoid sending any dummy data with keep-alive packets, no need to measure round-trip delay to lobby server, cos it's measured to zone server
tried:
- boost::asio::async_read(socket, boost::asio::buffer((void*)0, 0), handler) - causes "end of file" immediately
- boost::asio::async_read(socket, boost::asio::buffer((void*)0, 0), boost::asio::transfer_at_least(0), handler) - no "end of file" but handler is triggered immediately before any data was actually sent
trying to avoid sending any dummy data with keep-alive packets, no need to measure round-trip delay to lobby server, cos it's measured to zone server
TCP has no such thing as "empty packets" because TCP is a stream. If there is no data to receive, then there is no notification.
If you want packet semantics, you really want UDP, or perhaps (if you're adventurous and like the pain of early adoption), SCTP.
If you want packet semantics, you really want UDP, or perhaps (if you're adventurous and like the pain of early adoption), SCTP.
Quote:Original post by hplus0603
If they're all hitting a database all the time, sure!
If all they do is chat and occasional matchmaking, then, as Antheus says, you should be able to do 100,000 on a normal server machine with no sweat.It's more a question of what quality server bandwidth and hosting you want to pay for -- as in, how many players can you really afford? Technology is unlikely to be the limiting factor, unless you're at "real bussiness scale."
Ah, yeah, that'd be it. We do some pretty heavy database hitting with stuff at work (both Oracle and SQL Server). It runs on internal servers on the local network, but even then, pounding the database as often and as hard as we do, 5,000 clients would be a killer.
Author
Quote:Original post by hplus0603
TCP has no such thing as "empty packets" because TCP is a stream. If there is no data to receive, then there is no notification.
If you want packet semantics, you really want UDP, or perhaps (if you're adventurous and like the pain of early adoption), SCTP.
tnx, bth I had to realize this myself. it looks like TCP keep alive would be the most efficient way to detect broken connections. i guess it's handled on kernel level. not sure about linux, but on windows going from kernel level to user one and back are pretty time and resource consuming moves. but as you guys suggested, I don't need to worry about performance, so will do it as regular one-way dummy packets from server to client
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
