How to adjust the frequency of doing specific task in network update loop

Started by
9 comments, last by hplus0603 6 years, 12 months ago

Say if my game's network update rate is 20 times per second. But I want to do something else (maybe collision detection or sending extra stuff to client) at a lower frequency inside the network update loop, for example, 5 times or 4 times per second.

What is the best way to do this?

Should I use a global counter? Like this:

outside the loop:


int counter = 0;

inside the loop:


counter++;
if (counter % 5 === 0) {
    // do something
} 

I just wonder if things could go wrong when the server runs non-stop for a few months.

Advertisement
You wouldn't want counters like that, because they're unmanageable once you get a lot of them at different rates.
Just use timers. Set a future time for when you want something to execute, and when the current time is past that time, execute the thing, and set the timer to the next time you want it to execute.
Pseudocode:
float collisionDetectInterval = 0.25f; // 4Hz
float nextCollisionDetectTime = 0.0f; // do one immediately
while (1) // your loop
{
    if (getNowTimeInSeconds() >= nextCollisionDetectTime) 
    {
        DoCollisionDetect();
        nextCollisionDetectTime += collisionDetectInterval; // do another one later
    }
}
Most languages will have some object that can clean this sort of code up a bit.

You wouldn't want counters like that, because they're unmanageable once you get a lot of them at different rates.
Just use timers. Set a future time for when you want something to execute, and when the current time is past that time, execute the thing, and set the timer to the next time you want it to execute.
Pseudocode:


float collisionDetectInterval = 0.25f; // 4Hz
float nextCollisionDetectTime = 0.0f; // do one immediately
while (1) // your loop
{
    if (getNowTimeInSeconds() >= nextCollisionDetectTime) 
    {
        DoCollisionDetect();
        nextCollisionDetectTime += collisionDetectInterval; // do another one later
    }
}
Most languages will have some object that can clean this sort of code up a bit.

Thanks I would be using millisecond in my server, so I will change 0.25 to 250.

When it comes to uptime, milliseconds stored in a 32-bit integer will overflow after a while.
For a signed value, that happens at 24.85 days; for unsigned, it happens after twice as long.
It's better to store it in a 64-bit integer (long long, or int64_t.)
And, when you go 64-bit, you might as well use microseconds instead of milliseconds :-)

Note: None of the millisecond timers are accurate at all. timeGetTime() is the least inaccurate, but it's still quantized at several milliseconds. GetTickCount() and similar calls are often quantized to dozens of milliseconds.
On UNIX, use clock_gettime(CLOCK_MONOTONIC_RAW).
On Windows, use QueryPerfofmanceCounter() and divide by QueryPerformanceFrequency(). (Years ago, this counter would sometimes mysteriously "jump" but I know of no motherboard in the last 10 years who still have that bug.)
(There is also GetSystemTimePreciseAsFileTime(), but that has a lot more overhead, so you likely don't want to use that for game time progression.)
Using a "float" for time loses big, because it only has 24 bits of mantissa, and thus will suffer precision loss after less than a day.
Using a "double" for time theoretically has the same precision problem after a while, but that while is very long. If you "zero" the time when the server starts up, rather than dozens or hundreds of years in the past, "double" will probably work fine.

Best, however, is int64_t.
You figure out which step number your game is at by simply subtracting the start time, and dividing by the step progression rate:

int64_t game_start_time;

void start_game() {
  game_start_time = microseconds();
}

int current_step_number() {
  return (microseconds() - game_start_time) * STEPS_PER_SECOND / 1000000;
}
enum Bool { True, False, FileNotFound };

Using a "double" for time theoretically has the same precision problem after a while, but that while is very long. If you "zero" the time when the server starts up, rather than dozens or hundreds of years in the past, "double" will probably work fine.

I've worked with several engines that used double for seconds.

If you're operating at milliseconds, 2^53 mantissa is over a quarter million years before you hit catastrophic scale issues.

Trying to add something more useful, there are two major options.

One option is to add the time to the last update: nextCollisionDetectTime += collisionDetectInterval; This is useful because the next update will happen at regular times. If your system has variable timings it will continue at the regular step, running on the next possible update. This has a major drawback when you attach a debugger or do something that stalls, because now the system needs to update many times, maybe hundreds or thousands of times.

Another option is to specify a time in the near future: nextCollisionDetectTime = now + collisionDetectInterval; This is useful when you don't want the system to try to catch up. If your system has variable timings it will ignore skipped updates and send whenever time has elapsed from the last step. This has the opposite major drawback that actual intervals are irregular, cannot have position computed numerically, and will always drift to longer than the standard interval.

Both have useful scenarios.

Generally, I use the fixed-increment mechanism.

To make sure that the engine doesn't step too much when I attach with a debugger, I limit the number of steps that are allowed in a single go. If the difference is more than, say, 6 steps, I will hard-clip to 6 steps and slam the clock forward; this will of course cause network corrections if other hosts are involved.

int64_t current_physics_time = 0;
physics_state *prev_physics_state, *cur_physics_state;

void step_some() {
  int64_t now = microseconds();
  int steps = (now - current_physics_time) / STEP_INTERVAL;
  if (steps > 6) {
    steps = 6;
    current_physics_time = now - steps * STEP_INTERVAL;
  }
  while (steps-- > 0) {
    current_physics_time += STEP_INTERVAL;
    std::swap(prev_physics_state, cur_physics_state);
    simulate_one_step(); // reads prev state, writes cur state
  }
}

void display_frame() {
  int64_t now = microseconds();
  // note, as we interpolate between prev and current, this is one
  // frame behind leading edge time
  float alpha = (now - current_physics_time) / STEP_INTERVAL;
  display_interpolated_between(prev_physics_state, cur_physics_state, alpha);
}
The good thing about this is that you can separate your physics step time from your display rate time. For example, run physics at 120 Hz. Players with G-Sync displays or other high-rate outputs will see fine-grained stepping; players on Intel Integrated with a frame rate of 25 frames per second will typically step 5 physics steps per display frame.

If you display objects with interpolation, you'll typically want to display based on "now" offset back by one frame (see above.)
This, in turn, also works well for network extrapolation/interpolation, if you keep separate "current physics time" counters for each remote entity, or do client-side speculative simulation.
enum Bool { True, False, FileNotFound };

When it comes to uptime, milliseconds stored in a 32-bit integer will overflow after a while.
For a signed value, that happens at 24.85 days; for unsigned, it happens after twice as long.
It's better to store it in a 64-bit integer (long long, or int64_t.)
And, when you go 64-bit, you might as well use microseconds instead of milliseconds :-)

Note: None of the millisecond timers are accurate at all. timeGetTime() is the least inaccurate, but it's still quantized at several milliseconds. GetTickCount() and similar calls are often quantized to dozens of milliseconds.
On UNIX, use clock_gettime(CLOCK_MONOTONIC_RAW).
On Windows, use QueryPerfofmanceCounter() and divide by QueryPerformanceFrequency(). (Years ago, this counter would sometimes mysteriously "jump" but I know of no motherboard in the last 10 years who still have that bug.)
(There is also GetSystemTimePreciseAsFileTime(), but that has a lot more overhead, so you likely don't want to use that for game time progression.)
Using a "float" for time loses big, because it only has 24 bits of mantissa, and thus will suffer precision loss after less than a day.
Using a "double" for time theoretically has the same precision problem after a while, but that while is very long. If you "zero" the time when the server starts up, rather than dozens or hundreds of years in the past, "double" will probably work fine.

Best, however, is int64_t.
You figure out which step number your game is at by simply subtracting the start time, and dividing by the step progression rate:


int64_t game_start_time;

void start_game() {
  game_start_time = microseconds();
}

int current_step_number() {
  return (microseconds() - game_start_time) * STEPS_PER_SECOND / 1000000;
}

Thanks it leads me to think of using more accurate time for my game loop.

Do you even deem it a good idea to do hefty things like collision checking in the networking thread and to delay the networking thread to a rate?

I try hard to never (directly) delay/block the networking thread. It only ever delays when there is absolutely nothing to receive, nothing to send, and the timerfd is not ready either (timerfd assumes Linux, but can do the same with Windows). The timerfd is there solely to give the whole thing a notion of ticks. Everything received goes into the current bucket, and when the timerfd fires, everything received thereafter goes into the next. Somewhat just about the same as triple-buffering in graphics programming.

Each bucket thus contains one time quantum worth of inputs (or nothing, if no data came in) and produces one set of updates messages. But actually making something of what's in the buckets and returning something to send out is someone else's task, the network thread doesn't do that. As soon as there is a result ready to be sent, it is sent out immediately.

Maybe that's not the best approach, but it is something that in my opinion makes sense. You can only fit one datagram on the physical wire at a time, and receive buffers are small, so to keep latency low and to avoid packets being dropped, it's probably not a bad plan to try hard to never let it idle. That's my take, anyway. Might be wrong.

Do you even deem it a good idea to do hefty things like collision checking in the networking thread and to delay the networking thread to a rate?

I try hard to never (directly) delay/block the networking thread. It only ever delays when there is absolutely nothing to receive, nothing to send, and the timerfd is not ready either (timerfd assumes Linux, but can do the same with Windows). The timerfd is there solely to give the whole thing a notion of ticks. Everything received goes into the current bucket, and when the timerfd fires, everything received thereafter goes into the next. Somewhat just about the same as triple-buffering in graphics programming.

Each bucket thus contains one time quantum worth of inputs (or nothing, if no data came in) and produces one set of updates messages. But actually making something of what's in the buckets and returning something to send out is someone else's task, the network thread doesn't do that. As soon as there is a result ready to be sent, it is sent out immediately.

Maybe that's not the best approach, but it is something that in my opinion makes sense. You can only fit one datagram on the physical wire at a time, and receive buffers are small, so to keep latency low and to avoid packets being dropped, it's probably not a bad plan to try hard to never let it idle. That's my take, anyway. Might be wrong.

I think you have a point. Now I have a game physics update loop and a network update loop. Would you use a third loop to do the job? I have never written a MMO game before so I don't know which way is the right way. I do notice that the network update loop is too heavy duty, which may be the cause of making my game really slow with just a few dozen players.

MMO game
That is a huge beast. A MMO is not something you just write on a couple of afternoons. Or, as single developer, for that matter. Or, without a lot of experience.

MMO game normally means a topology of externally visible nodes (both for bandwidth and load balancing) and internal servers with external nodes doing the crypto/correctness lifting and otherwise act as both publishers and subscribers, pretty dumb otherwise (except for caching everything that can be cached, they're pretty dumb, they don't do much "game stuff"). Some internal server or servers, in addition to a dedicated database and an authentication server etc., act as subscriber to what the end nodes publish, and in return publish what the end nodes need to know about what happened. They'll let the clients know.

The reason for this is that you will usually have many clients (a few hundred = nothing) and a rather huge world, and need to have some kind of spatial subdivision or region of interest stuff. Otherwise, you're dead. Even if you only ever send an update when something really important happens (player 6340 shoots at player 7343), you cannot possibly send all updates to all clients. That, and you need to be able to scale, because no matter what you think the size of your game or the required resources will be, your estimate will always be wrong. Scale well, or die.

Using PUB/SUB works, and is vastly independent of the number of computers on either end of the "pipe". Also, you need to somehow aggregate messages from different users that are quite possibly connected to different external nodes but whose avatars are in the same spatially located bin. Everyone in A town will be handled by server A and everyone in B town will be handled on server B. Somehow, you must get the correct messages to the correct server, and it must still work if you later decide that you actually don't need server B, or server A has nearly no load and could as well handle town C in addition. Or, if you realize that you need another three servers.

The publisher/subscriber model addresses these problems, and readily working, robust implementations exist.

So basically, you have two very different servers. One of them:

  • lets clients connect, checks that the token they provide is recognized by auth server as token from successful login
  • sets up secret keys, whatever
  • receives packets, decrypts, decompresses, does basic correctness/plausibility checks, times out disconnected clients
  • publishes to the correct subscription channel (that means it must have an idea whereabout in the world the user is)
  • subscribes to update message channel in region of interest (that is, region of interest for each connected client)
  • receives messages, send out updates as they are available (but only the ones of each single client's region of interest, sorted by importance, and rate-limited to whatever max rate you can afford, possibly dropping soon-to-be-invalidated stuff that won't fit in your quota)
  • no delay whatsoever, except when waiting for messages (or user input) to come in

The other one could do something like:

  • subscribe to all channels (or all channels on one "continent" or "city" or whatever)
  • receive messages, place all in one bucket until "tick", then increment the bucket
  • in a pool of workers, work off events per-channel, aggregate new world state (do not modify current state!), generate per-channel updates, attach an "importance" metric. Flip states. This is "the actual game".
  • publish updates, no delay

Something different is possible. Instead of processing events per sub-region of interest (that seems practical since events are already sorted that way!) one could, within one larger region's scope (i.e. within one physical server) just iterate linearly over the world state which is arguably more cache efficient, and since even inactive players will usually have something happen (regenerating, cooldowns) it's probably not a wrong approach to just iterate over them all linearly. This also parallelizes very nicely without needing much synchronization.

This topic is closed to new replies.

Advertisement