Sign in to follow this  
Demx

select() in server

Recommended Posts

Demx    389
IS it better to use select() and synchronous sockets to make a 2D online game server, rather than many threads per client ?

Share this post


Link to post
Share on other sites
swiftcoder    18426
[quote name='Demx' timestamp='1350568496' post='4991428']
IS it better to use select() and synchronous sockets to make a 2D online game server, rather than many threads per client ?[/quote]
Generally, yes. Better yet would be one of io-completion-ports/epoll/kqueue, or a [url="http://libevent.org"]wrapper thereon[/url].

Share this post


Link to post
Share on other sites
Demx    389
is [url="http://www.codeproject.com/Articles/20066/A-scalable-client-server-using-select-socket-funct#_comments"]http://www.codeproject.com/Articles/20066/A-scalable-client-server-using-select-socket-funct#_comments[/url] a good base on which to start ?

Share this post


Link to post
Share on other sites
swiftcoder    18426
[quote name='KnolanCross' timestamp='1350607314' post='4991608']
When I had to choose I went for multiple threads model because most cpus nowadays are multi-core, so a multi-threaded application is more likely to scale better.[/quote]
There is no scaling if all you are doing is using threads to implement non-blocking I/O. Now, if you are also processing data in each those threads, well, that's a different story (with its own pros and cons).

Share this post


Link to post
Share on other sites
KnolanCross    1974
[quote name='swiftcoder' timestamp='1350607655' post='4991609']
[quote name='KnolanCross' timestamp='1350607314' post='4991608']
When I had to choose I went for multiple threads model because most cpus nowadays are multi-core, so a multi-threaded application is more likely to scale better.[/quote]
There is no scaling if all you are doing is using threads to implement non-blocking I/O. Now, if you are also processing data in each those threads, well, that's a different story (with its own pros and cons).
[/quote]

Well observed.
You should be using a blocking interface along with a timeout (such as set_sock_opt) or a single file descriptor select (in my case) so you can update each thread and also have them in the sleep process most of the time.

Share this post


Link to post
Share on other sites
hplus0603    11347
If you ever find yourself needing to sleep a thread, you're doing something wrong. You should be using blocking I/O (and other primitives, like condition variables, events, etc,) and be driven by that. If your OS has no native unblocking timers, and you need to step at a fixed rate, it's OK for one thread to use a timeout-wait for blocking primitives to implement that.

If you use thread-per-client for "scaling," exactly what is each of those threads doing? How are you avoiding those threads stepping on each others toes while mutating your world?

Share this post


Link to post
Share on other sites
KnolanCross    1974
[quote name='hplus0603' timestamp='1350621967' post='4991674']
If you ever find yourself needing to sleep a thread, you're doing something wrong. You should be using blocking I/O (and other primitives, like condition variables, events, etc,) and be driven by that. If your OS has no native unblocking timers, and you need to step at a fixed rate, it's OK for one thread to use a timeout-wait for blocking primitives to implement that.

If you use thread-per-client for "scaling," exactly what is each of those threads doing? How are you avoiding those threads stepping on each others toes while mutating your world?
[/quote]

By sleep i didn't mean the sleep function, I meant the OS' sleep state.

In my case I had threads to receive info/send updates to clients and threads to update the world state. To avoid running conditions I used a lot of mutexes and a zone approach to filter data (data in a zone could be changed without affecting others). It was indeed pretty annoying to implement it, but I learned a lot. How he will model his world is up to him [img]http://public.gamedev.net//public/style_emoticons/default/happy.png[/img] Edited by KnolanCross

Share this post


Link to post
Share on other sites
hplus0603    11347
[quote]By sleep i didn't mean the sleep function, I meant the OS' sleep state.[/quote]

It may be a terminology thing. The way I understand it, a thread enters "sleep" state by calling sleep() or similar functions. This is different from the "blocked" or "wait" state which is when a thread is actually waiting on something else to happen.

Also, if you use mutexes in your threads, you won't actually scale across cores, because your threads will end up serializing on whatever the common data structure is. Thus, you can save memory, CPU overhead, and programming complexity by using select() or non-blocking sockets instead of multiple threads for your networking.

Share this post


Link to post
Share on other sites
KnolanCross    1974
[quote name='hplus0603' timestamp='1350664255' post='4991815']
[quote]By sleep i didn't mean the sleep function, I meant the OS' sleep state.[/quote]

It may be a terminology thing. The way I understand it, a thread enters "sleep" state by calling sleep() or similar functions. This is different from the "blocked" or "wait" state which is when a thread is actually waiting on something else to happen.

Also, if you use mutexes in your threads, you won't actually scale across cores, because your threads will end up serializing on whatever the common data structure is. Thus, you can save memory, CPU overhead, and programming complexity by using select() or non-blocking sockets instead of multiple threads for your networking.
[/quote]

Yeah, it is terminology, afaik even the sleep call is an io block state, waiting for the os to "wake" the process, for instance sleep (10) will make your program stay in the wait state for 10 seconds, while a blocking receive will be in wait state until new data arrives. Anyways, I meant that the process would be in a blocking state and would not consume CPU.

About the mutexes I can see your point but I respectfully disagree.
Mutexes are not active wait, in other words when you reach a locked mutex, your thread enter a wait state, thus it won't consume cpu. In my case, although I had some threads locked in the same mutexes, there were others that were not. So I was using the multiple CPU cores at the same time, this is a huge gain in performance.
There is, of course, an increase use of memory and there is an overhead for the scheduler (since there were multiple threads). Edited by KnolanCross

Share this post


Link to post
Share on other sites
BeerNutts    4400
IMO, you're going to cause more pain than needed for using multiple threads for each client's connection, and for little (if any) benefit.

You could create 1 thread to handle all client connections, and just check all the clients sockets with select, or, you could treat the client connections portion as you would any other part of a typical game loop update:

[code]
while(IsRunning) {
UpdateClients();
UpdateWorld();
// etc...
}

void UpdateClients() {
struct timeval timeout;
timeout.tv_sec = 0;
timeout.tv_usec = 0;
// assuming you modify the file descriptor on client connect and disconnect...
int clientsReady = select(NumClients, &ClientFDs, NULL, NULL, &timeout);
if (clientsReady > 0) {
// process the clients
}
else if (clientsRead == -1) {
// handle error
}
}
[/code]

Share this post


Link to post
Share on other sites
Katie    2244
"When I had to choose I went for multiple threads model because most cpus nowadays are multi-core, so a multi-threaded application is more likely to scale better."

You'll still need to use a multiplexing. Why? Because you don;t want the threads (and hence their attendant data structures) being scheduled all over the place. If you have more threads than cores, you run the risks of either threads being runnable but not running (because they're current assigned to cores which are already running something else), or being moved to another core requiring a lot of memory controller bandwidth (which slows down your thread working).

Ideally, you should be looking to have just less threads runnable at the same time than cores; leave a couple of cores for the OS and generally it'll happily settle into the gap and not thrash with you. This means your threads exhibit effective soft core affinity but without all the aggravation that real affinity brings (presuming your OS will even let you do it).

Share this post


Link to post
Share on other sites
KnolanCross    1974
I believe this may be going away off-topic... again, I will respectuffly disagree (and by that I mean: I can see your point, they are correct, but I put different weights on the advantages/disavantages than you).

1) Schuduling multiple threads is a only problem when they are all working non-stop. It is not the case here, since each thread will work for a very short period of time and get back to the wait state.

2) Didn't really got the problem of falling on different core. The only problem I can see is a cache miss, but if you are working with a single core and a lot of data, you are very likely to face this problem anyway.

3) Still on the different core issue. A single process approach will enter on IO wait in the select and may also be processed by a different core each time.

I would like to point again that I am not saying that this is the mathematically proven best approach or that it won't run into problems. The way I see it, the way OS' handle multi threading will improove a lot in the next years and CPUs will have more and more cores, so this would be the way to go (not only for having a better application, but also to gain the knowledge about it). Edited by KnolanCross

Share this post


Link to post
Share on other sites
[quote name='BeerNutts' timestamp='1350674556' post='4991876']
IMO, you're going to cause more pain than needed for using multiple threads for each client's connection, and for little (if any) benefit.

You could create 1 thread to handle all client connections, and just check all the clients sockets with select, or, you could treat the client connections portion as you would any other part of a typical game loop update:

[code]

int clientsReady = select([b]NumClients[/b], &ClientFDs, NULL, NULL, &timeout);
if (clientsReady > 0) {
// process the clients
}
else if (clientsRead == -1) {
// handle error
}

[/code]
[/quote]

numClients is a bit wrong here. The first argument shall not be the number of fds in your fdset but the number of the highest fd to check + 1.

[url="http://linux.die.net/man/2/select"]http://linux.die.net/man/2/select[/url]
nfds is the highest-numbered file descriptor in any of the three sets, plus 1.

Its also worth mentioning that ClientFDs and timeout should be reinitialized after select.

To answer the question:

Go for async IO and use select/kqueue/poll or whatever else you like. select is pretty easy to use , even tho a bit confusing at first. especially if you want to to asyncronous write operations as well or add fds from files to your select queue to do asynchronous file access to the OS. But i know from back in the days that UOX2 (a very famous Ultima Online freeshard) was coded with select and ran great. Later one they added threads etc to leverage level loading and other stuff but in the beginning everything was one thread and async IO. I think thats anyway the way to go for online rpgs. You should just take care not to do put too much stuff in each cycle of the main loop. But thats where sharding comes into the picture. For example, Ultima Online (the real one), had multiple servers per world. Edited by FlyingDutchman

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this