game server threads (very high level design)
Author
I've been contemplating threading parts of my game server, but I am not exactly sure about the best way to go about it.
MODEL #1 -- I first considered having game logic and connection producer be the "main thread" and having each connection be its own individual thread. (So, 100 active players would result in 101 threads total running.) I eventually shied away from that because I did not like the idea of being uncertain about which connection is running when.
MODEL #2 -- My second design involved having two threads: the main game logic thread and the connection handler thread. The connection handler thread would just run in circles checking each connection for updates, checking for new connection requests, etc.
Any suggestions on which direction is superior? Or are they the same with just different implications? Or is there a third model I don't know about? :P
Just to give you an idea (I'm in no way an expert on the subject):
The problem with the first idea is that you will have a lot of threads if you have a lot of players. This could be a problem, but that depends on the game. If you never have more than say, 16 players connected it won't be a problem.
The problem with the second idea is when something goes wrong with the connection handler thread, you have a problem.
If I would be in your situation I would have a number of connection handler threads (like in your second idea), each one doing the same thing, but not as much as one for each connection. That way, if something goes wrong in one thread, there's still other threads doing work and you have time to start a new one. You could have all the connected clients in a queue and every one of the connection handler threads pops one from the queue, handles it and pushes it in again at the back.
The problem with the first idea is that you will have a lot of threads if you have a lot of players. This could be a problem, but that depends on the game. If you never have more than say, 16 players connected it won't be a problem.
The problem with the second idea is when something goes wrong with the connection handler thread, you have a problem.
If I would be in your situation I would have a number of connection handler threads (like in your second idea), each one doing the same thing, but not as much as one for each connection. That way, if something goes wrong in one thread, there's still other threads doing work and you have time to start a new one. You could have all the connected clients in a queue and every one of the connection handler threads pops one from the queue, handles it and pushes it in again at the back.
If it matters depends on the game. If you are talking about a multiplayer game where expected player counts is under 100 then go with the first option it will be easier. If you are talking about an MMO go the second route as the first one won't scale(1mb of stack space per thread eats a lot of ram, 2K players means 2GB ram for their connection threads, not including any dynamic memory or other per player overhead.)
Author
Quote:Original post by stonemetal
If it matters depends on the game. If you are talking about a multiplayer game where expected player counts is under 100 then go with the first option it will be easier. If you are talking about an MMO go the second route as the first one won't scale(1mb of stack space per thread eats a lot of ram, 2K players means 2GB ram for their connection threads, not including any dynamic memory or other per player overhead.)
Ah... I did not know about that. Thanks for the heads up. The more I think about it, the more I like my model #2. And yeah, I was kinda leanin' in the MMO direction, so having one master update thread seems to be the way to go.
Basically, I want the connection thread to run down all the connections, receive data, put that data onto a "world stack" of sorts, send out updates to the clients, and then push that world stack into the main thread somehow (perhaps some mutexed update stack).
I'm just thinking in general terms.
Quote:Original post by Schildpad
If you never have more than say, 16 players connected it won't be a problem.
It depends on what you would consider a "problem". Having a lot more threads than processing cores (which in this scenario is likely; how many people have an eight-core hyper-threading CPU?) will likely result in worse performance than if you matched thread count to core count. Hence, the problem with spawning one thread per connection is that you can't control the amount of threads created. As others have pointed out this is definitely something to avoid if aiming for player counts in the hundreds. A project I used to work on did that, and wouldn't scale beyond some 100 users even if the hardware was reasonably capable. The server was eventually rewritten using Boost.Asio, which has enabled it to handle hundreds of players.
I was just dealing with this exact issue for a system I was writing in work (telco infrastructure stuff), in Java (basically, io vs nio) and heres what I found:
* The threaded model is simpler to code for
* Thread overhead isn't as bad as one would suspect
* The asynchronous model isn't faster, but it does scale to larger numbers of connections
For my code, I will be using the thread model, because I don't need thousands of connections and the simpler code is worth the loss of scalability.
But, if I were in your position, I'd probably have a single thread sending and receiving data asynchronously, have an input and an output queue and a thread pool to process the data. So, each thread in the pool would take the next item off the input queue, process it and put the result (if any) on the output queue. The networking thread would block until theres input to read from one of the connected sockets, or until theres something on the output queue. The thread pool size would be automatically set from the number of available processors/cores.
* The threaded model is simpler to code for
* Thread overhead isn't as bad as one would suspect
* The asynchronous model isn't faster, but it does scale to larger numbers of connections
For my code, I will be using the thread model, because I don't need thousands of connections and the simpler code is worth the loss of scalability.
But, if I were in your position, I'd probably have a single thread sending and receiving data asynchronously, have an input and an output queue and a thread pool to process the data. So, each thread in the pool would take the next item off the input queue, process it and put the result (if any) on the output queue. The networking thread would block until theres input to read from one of the connected sockets, or until theres something on the output queue. The thread pool size would be automatically set from the number of available processors/cores.
Author
I'll just have to try a few things and see what works best. I'm more a fan of synchronous non-blocking updates over starting tons of asynchronous updates.
Perhaps you missed the "very high level design" part, but I'll humour you and say that I am using C++ in Linux.
Quote:Original post by Antheus
Which language? Which OS?
Perhaps you missed the "very high level design" part, but I'll humour you and say that I am using C++ in Linux.
Quote:Original post by TheBuzzSaw
Perhaps you missed the "very high level design" part, but I'll humour you and say that I am using C++ in Linux.
The very high level design depends on available APIs and characteristics of the system.
C++ and Linux: boost::asio. Alternatively, libevent.
Proactor pattern, event driven, constant number of worker threads. Network events are multiplexed across multiple threads, simulation and other systems run as single-threaded, in any available thread.
This design scales to tens of thousands of connections, ensures responsiveness and has generally good scalability characteristics.
Unfortunately, high-level design on its own is not enough, which is why specific mention of ::asio library. There are many implementation details, especially regarding networking, where there are many competing APIs of varying availability and quality.
In addition, ::asio solves some very common problems that networking APIs on their own don't provide, such as consistent handling of timeouts and cooperative timer support, support for global/non-reentrant code and typesafe inter-thread communication.
Author
My instinct tells me that I should fear Boost.Asio since it appears to still be version 1.0 (am I looking at the wrong page?), but I've only heard good things about it. I'll give it a try.
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
