So, I've managed to build a pretty decently working server software for myself which implements a myriad of techniques like RPC, Events, State Synchronization, all in all I'm pretty damn proud, and I have worked around the issue I'm going to ask about here, but I'm not sure I'm satisfied with the solution (which, by looking at past experiences, often means it's not optimal).

The server is running, everything is fine, it's handling different rooms, clients, actors over the network. But when everyone disconnects, or there are very few players on the server, I don't want it to just spin through receive->simulate->send loop over and over again because this basically steals one core (the server is single-threaded) at 100% usage and with the server just spinning round round round.

So I solved this by a few clever checks with hunts down any remaining work, and when we're sure there is absolutely NO work to do, it will stall for 1ms by doing sleep(1). This feels very very dirty (I've done a decent chunk of multi-threaded applications, and sleep() is usually the effect of a somewhat bad design). I've also tried with using several different types of the thread/os-level events available in Win32, timers, etc. but nothing seems to be working as well as I want.

I've been looking at the functions in winmm.dll which contains a few functions for very high accuracy timers (at least compared to Sleep() and the other various types of thread scheduling functions), so maybe this is a the way to go?

How is the state of "I have nothing to do right now" usually dealt with in game servers?

[quote name='fholm' timestamp='1319047054' post='4874395']
So I solved this by a few clever checks with hunts down any remaining work, and when we're sure there is absolutely NO work to do, it will stall for 1ms by doing sleep(1). This feels very very dirty (I've done a decent chunk of multi-threaded applications, and sleep() is usually the effect of a somewhat bad design). I've also tried with using several different types of the thread/os-level events available in Win32, timers, etc. but nothing seems to be working as well as I want.

If it's really sleep(1) (from the C standard library) then that will sleep for 1000 milliseconds. If it's Sleep(1) (from Win32) then it's a millisecond, but you probably should be using SleepEx().

The way this is typically done in servers is to build the server as reacting to events. The events you have available are "time to step the world" (queued N times per second), "new user connected," "existing user dropped" and "new data available from existing user."

If this is also a Windows GUI application, then you also need to worry about the Windows event loop, which means that you need to wake up when there's a message waiting in the queue.

For up to 64 sockets, you can use MsgWaitForMultipleObjectsEx(). The loop looks something like:

forever() {
while (PeekMessage()) {
Translate/Dispatch/DialogMessage();
}
DealWithAnyNetworking();
sleepTime = TimeOfNextWorldStep() - TimeNow();
MsgWaitForMultipleObjectsEx(All Sockets, sleepTime);
}


In the end, you may want to factor "timeofnextworldstep" into a time-based priority queue, call it a timer queue or event queue.

If you have more than 64 sockets, you will need to use I/O completion ports, and native Windows timers, which, as you've discovered, aren't super reliable. You can, however, set a timer on a window to run every 10 milliseconds, and each time it runs, check to see if it's time to step the world, and do so, and the jitter won't be too bad. You can step more than one step at a time if you're falling behind real time.
[/quote]

Thanks for your input! I am using I/O Completion ports for my socket stuff, I found one native windows timer (of the like 20 there is ;p) that actually seems to be accurate enough, I need down to 20ms and up to 200ms simulation accuracy (with fixed 10ms increments), which is CreateTimerQueueTimer from kernel32.dll. The IO is received and sent in a secondary thread, but the actions taken from the IO is done in the main thread of course. Basically have two threads running: Main Thread, IO Thread. I managed to throttle the main thread to 20-200ms using CreateTimerQueueTimer, and then the IO thread works on the events from the underlying I/O Completion ports. The IO thread prepares the input for consumption by the main thread and puts in a message queue, when the main thread needs to send data back to a client it puts in on an output queue that the IO thread reads from.

I also keep track of the total amount of elapsed milliseconds, and before every step I do "expectTicks = elaspedMS % timeStep", if expectTicks is larger then currentTicks+1, I will just run the simulation as many times extra as I need to.

The server is running, everything is fine, it's handling different rooms, clients, actors over the network. But when everyone disconnects, or there are very few players on the server, I don't want it to just spin through receive->simulate->send loop over and over again because this basically steals one core (the server is single-threaded) at 100% usage and with the server just spinning round round round.

So I solved this by a few clever checks with hunts down any remaining work, and when we're sure there is absolutely NO work to do, it will stall for 1ms by doing sleep(1). This feels very very dirty (I've done a decent chunk of multi-threaded applications, and sleep() is usually the effect of a somewhat bad design). I've also tried with using several different types of the thread/os-level events available in Win32, timers, etc. but nothing seems to be working as well as I want.

If you are using IOCP on Windows, then something is wrong with your current implementation because IOCP naturally solves this issue by design. In typical IOCP use, you create a bunch of network worker threads that all block on [font="Consolas, Courier, monospace"]GetQueuedCompletionStatus[/font], waiting for work. When there is no work, there is no worker thread execution because they are all blocking. That means nothing will be spin waiting or wasting CPU cycles in an infinite loop bounded by a Sleep/Ex call. The scalability aspect of IOCP comes from the fact when you need more processing power, you simply create more threads that sit on GQCS and assuming you have the hardware resources, everything just works without change to anything else (of course, you have to have designed code that scales, but that is another issue).

If you have all your worker threads blocking on GQCP, the most obvious question is how do you exit worker threads? This is also really easy by design, since you just call [font="Consolas, Courier, monospace"]PostQueuedCompletionStatus[/font] with a user defined message that you process in the worker thread that signals the worker thread to exit and not loop back to the next call to GQCP. Likewise, for any other custom event handling, PQCS is used to give the next available worker thread work to do.

To implement a generic worker thread event handling system, you could also make use of the [font="Consolas, Courier, monospace"][/font][font="Consolas, Courier, monospace"]QueueUserAPC [/font]function. In that design, you simply setup a pool of worker threads that all block infinity with a SleepEx call. When you have work to process, you call QUAPC with one of the handles of the worker threads so the work is processed in context of that thread. For more information on that topic, check out the Parallel Programming with C++, an older, but still revelant blog entry that has very useful information about some Win32 stuff.

I wrote some stuff with IOCP and UDP a while ago that sort of pertains to these issues. I have learned quite a lot since, so some stuff I got right, but a lot of stuff I got wrong with my understanding (too much to go over). Nowadays, unless you absolutely have to stick to writing everything yourself, there are libraries out there that take care of all these things for you and make life a lot easier. Such libraries include ACE and Boost. Boost::Asio is the main library you would be looking to get into, but there are a lot of other Boost libraries you would make use of. I also wrote some boost::asio stuff not too long ago. Once again, some stuff I got right, some stuff I got wrong, so take it with a grain of salt. There is a lot of issues at hand that would have to be explained first before simply updating those works.

Anyways, getting back to the issues at hand, taking care of the IOCP aspect will fix anything inefficient about the network related work. What it will not take care of is your main simulation loop that performs any system upkeep. You can either stick with the Sleep pattern to throttle when there is no work or you can redesign your system to be more scalable and asynchronous and make use of worker threads to handle execution of any pending work.

For example, using boost, you would have at minimal two thread pools of worker threads. The first pool is for network related stuff only. The callbacks that execute should involve very little shared state so you should not be hitting any unavoidable bottlenecks from having to synchronize access to global shared resources. This means that most business logic processing does not take place in these threads, work is forwarded outside that system. The second worker thread pool is for everything else that needs to execute. The reason for separating them is to achieve a more flexible system rather than having to worry about longer running tasks from starving out other critical tasks. Such a design is taken by .Net in their ThreadPool design along with Async operations.

In doing so, all threads are always blocking, waiting for work. When there is no work, there is no execution so you do not waste any resources. When there is work, then threads will process it as efficiently as your code is implemented and that is that, The only times you might need to use Sleep in such a design is when the overhead of starting a new async operation is higher than simply waiting and trying again for a limited number of times. For example, say you need to open a file. If your first attempt fails, you might want to loop a few times and keep trying rather than simply let the operation fail. Otherwise, you would be setting up a timer to execute a limited number of times and then have to continue execution when the file was opened that really complicates things.

The downside to using boost is the resource/performance overhead you are trading for simplicity and convenience. In most cases, it is not a big deal, because you end up writing easier to maintain and understand code without having to worry about all the little annoying issues you otherwise would have to. However, some people have very specific needs and are unable to use such libraries due to licensing issues, but if that does not apply to you, then you should definitely check it out. That is not to say boost is not without its own quips, but there is a large community of support available.