Sign in to follow this  
Synex

How many threads?

Recommended Posts

Synex    170
I've done a bit of multi-threaded programming in the past, but only of a very small scale. I'm busy beavering away at a game server that will serve quite a few clients. I'm just wondering how many threads can i get going at once? I realise there is probably no limit, but i'm wondering what a good theoretical limit would be. At the moment, i've got a few different thread layouts in mind: A: 2 threads per client. One send / one receive thread. Messages get put into a message store and dealt with by the main server thread. B: Multiple threads per client. Send / Receive threads, plus seperate threads for each process it asks for. Therefore, a simple 'echo' request to the server will be dealt with instantly, while longer requests such as information from the database can run in the background / files being sent from server > client. C: Thread pool. Say 20 threads running, getting handed as / when they are needed. However, if all 20 threads get bogged down doing something complicated (sending a file, large database query) then the clients will all get lagged out. Any ideas? How do the rest of you do it? What is the theoretical maximum threads you'd use? Cheers,

Share this post


Link to post
Share on other sites
Catafriggm    296
Thread pool is really the only acceptable option. You might be interested in completion ports, as well. As for the number of threads, you'd want ideally the same number of threads running all the time as the number of CPUs. But you'll have to adjust for how much time the threads spend waiting (if they're doing synchronous I/O or something, you'll want more threads than the number of CPUs). But it should be possible to calculate the right number, based on profiling. If in doubt, you could always use the rough estimate of 1.5x as many threads as CPUs.

Share this post


Link to post
Share on other sites
pragma Fury    343
I agree, a thread pool is probably the 'goodest' option here. Set the number of threads at 1-2x the number of CPU's and fine-tune it from there.

As soon as you start getting >100 threads, you're really going to see a lot of performance issues. And I think there is a per-process thread limit, though I don't know what it is.

Share this post


Link to post
Share on other sites
mgarriss    154
actually there is a limit to the number of threads you can use. it's different for each OS. for example, i ran in the a rather low limit on HPUX once.

the number of threads you want/need is not really related to the number of CPUs. like you have pointed out most threads will be blocking on a resource (a TCP read for example).

it sounds like you have a good understanding already of what you need.

p.s. i highly recommend boost threads if you are not already using them. they are very slick, very well tested, and include most of what you want to do with threads (condition vars, mutexes, etc).

_ugly

Share this post


Link to post
Share on other sites
Catafriggm    296
Quote:
Windows NT introduced kernel support for notification ports (called I/O completion ports) in version 3.5. While possible to implement a message queue (or to marshal I/O completion notifications to a message queue) in user mode, doing it in the kernel gives some very nice features, such as optimization of CPU use. It's usually considered a good idea to create more threads in a thread pool than CPUs in the computer. The reason is that the threads may go into wait for something not related to the notification port, such as doing synchronous I/O while processing a notification. If there are more threads than CPUs, that ensures that even if some threads go into wait, other threads will be ready to keep all CPUs busy.

The problem with that is that it is wasteful in a different way. It's moderately expensive for a CPU to stop working on one thread and start working on another - this happens every time a thread's time-slice expires. If there are more threads running than CPUs to run on, this will happen more often - every 60 to 120 ms, on Windows NT. I/O completion ports were designed to solve both problems, by tracking (in the kernel) exactly when threads go into wait. If one thread from the notification pool goes into wait while processing a notification message, another thread will be run from the pool; yet no more threads in the pool than CPUs will be allowed to run concurrently. This is possible because the kernel knows exactly how many threads are running, and how many are waiting; this is something that can't reasonably be done without kernel support.

Share this post


Link to post
Share on other sites
OuncleJulien    157
I think you may want to look into connnection pooling with the select function. It allows you to watch multiple sockets at once in a single thread. If you also put your servers socket in the select you wouldn't need any threads. In order to do this you'd need to set all sockets to non-blocking so the server doesn't hang. It works really quite well if implemented right.

Share this post


Link to post
Share on other sites
pragma Fury    343
FYI, Using the following bit of code:

for(int n=0; n < 1000000; n++)
{
if(!CreateThread(NULL,0,ThreadProc,0, CREATE_SUSPENDED,NULL))
{
printf("Failed to create thread #%d",n+1);
break;
}
}





I find that the limit (on my computer) is 2033 threads (including the process thread). I ran it a couple times and got that number back. Dunno if it's useful info or not..

Edit: Got the same number on a dual-xeon box.

Share this post


Link to post
Share on other sites
Catafriggm    296
Not really useful. If you try to use that many threads in a process, you can go outback and shoot yourself :P Even in the most extreme of cases a process shouldn't be using more than 30 or 40 threads. Anything more is deplorably wasteful.

Share this post


Link to post
Share on other sites
pragma Fury    343
Quote:
Original post by Catafriggm
Not really useful. If you try to use that many threads in a process, you can go outback and shoot yourself :P Even in the most extreme of cases a process shouldn't be using more than 30 or 40 threads. Anything more is deplorably wasteful.


Oh I agree. I knew the limit was around 2000 before the OS stopped you, I just wanted to find out :)

Share this post


Link to post
Share on other sites
Synex    170
Ok. Thanks for the quick response, some really useful points here.

First off, on the CPUs front... this server will likely be run on single processor systems. The idea is for players to be able to set up their own servers like Freelancer etc that other people can connect to, and i dont know of many people dedicated enough to have a dual processor system. Therefore, all threads will be run on the same CPU.

Thread Pooling - looks like i'll go with this one. I'll try and work out the optimum number of threads, perhaps spliting services by threads (IE chat messages and other 'quick' things to do get stuck in one thread, heavy database calls in another thread etc.) I need a multi-threaded model to keep the response time on the client low, allowing you to keep chatting to your friends while the client is downloading game data.

An example of this would be talking while downloading a map. The server would freeze up everytime someone requested a file because itd be busy sending off that file and not doing anything else. Therefore, one thread can be busy throwing files out all over the place, while another thread is receiving and sending off chat messages to all the clients etc. I suppose, by spliting different services into different threads would remove the problem of multiple threads trying to access the same resource as well (multiple threads writing to the chat log for example).

I'll take a good look at Async sockets and connection pooling and let you all know how it goes.

Thanks

Share this post


Link to post
Share on other sites
doho    378
You need as few threads as possible unless there is a really good reason. Couldn't you use a simple webserver to give the functionality of downloading maps without disturbing your game? It surely seems alot easier.

Share this post


Link to post
Share on other sites
Synex    170
Quote:
Original post by doho
You need as few threads as possible unless there is a really good reason. Couldn't you use a simple webserver to give the functionality of downloading maps without disturbing your game? It surely seems alot easier.


Certain of the game data is static, so gets downloaded onto the client from the server automatically when you connect (map details, item details etc.)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this