Jump to content
  • Advertisement
Bow_vernon

Confirmation on gaffer's state synchronization method

Recommended Posts

Hi folks, been years since last time I posted here. I've made a multiplayer shooter in the past, following the valve's networking model. Now I want to try to make a multiplayer sim racing, and got the physics part done. I've read the article on gaffer's networked physics. I've also watched his GDC presentation. However, nowhere in his materials did he say that there is some kind of physics rewinding. I am trying to follow his third method, "state synchronization". I wonder what did he do when the client receive server's snapshots (consists of states + input, not just state like in interpolation method)? He didn't mention to rewind the physics to the server's snapshot and replaying all inputs up to current client's tick. Or did I miss something? Cause if there's no rewinding involved, that'd be great and simplify a lot of things in the netcode. (I am aware that at his second method, "snapshot interpolation", there is no simulation at all for remote objects. But I feel that'd be not too suitable for racing games cause added delay is quite noticeable).

Share this post


Link to post
Share on other sites
Advertisement

I forget which version the "third" version is, but is that perhaps the version that requires a full round-trip to the server before you see the movement? Or is it the version where actors aren't allowed to interact/collide with each other directly? In either of those cases, you won't run into cases where the server simulates differently from you, other than if your simulation itself is nondeterministic. In those cases, you have the option of backing up and re-simulating each time a snapshot is received, or applying additional "drift" to the simulation model to compensate for whatever differences you detect between local and server states.

Share this post


Link to post
Share on other sites
On 1/26/2019 at 12:37 PM, hplus0603 said:

I forget which version the "third" version is, but is that perhaps the version that requires a full round-trip to the server before you see the movement? Or is it the version where actors aren't allowed to interact/collide with each other directly? In either of those cases, you won't run into cases where the server simulates differently from you, other than if your simulation itself is nondeterministic. In those cases, you have the option of backing up and re-simulating each time a snapshot is received, or applying additional "drift" to the simulation model to compensate for whatever differences you detect between local and server states.

the third version is state synchronization method, where he let the client simulate along with the server. However I think I got this, but needs confirmation. In the article he stated that we need to make a jitter buffer for the response from the server. But that means the client is simulating behind the server right?

Also I wonder what kind of synchronization method was used in gta 5 online. It seems perfect, even when racing, there seems to be no problem like in the F1 2017 multiplayer where there would be usually "ghost" collision (i.e., you got hit by someone behind you who was actually far way baack in your screen).

Share this post


Link to post
Share on other sites

Can you link to which of gaffer's work you're following?

On 1/25/2019 at 2:34 PM, Bow_vernon said:

However, nowhere in his materials did he say that there is some kind of physics rewinding. I am trying to follow his third method, "state synchronization". I wonder what did he do when the client receive server's snapshots (consists of states + input, not just state like in interpolation method)? He didn't mention to rewind the physics to the server's snapshot and replaying all inputs up to current client's tick.

I'm not sure if he does this (I'll have to re-read his work :)) but that's exactly what we do when using synchronized physics simulation in our racing game. It does add a lot of extra CPU load on the clients, especially at high ping...
In the "Counter Strike" model, the reverse happens -- the server pays the cost of constantly rewinding / fast-forwarding.

Also, just for clarity, the clients actually have to simulate / extrapolate to a point in time that's at least ping*0.5ms ahead of the server clock -- so that their input commands will arrive at the server just before the point in time when they should be applied to the simulation. We do a clock-synchronization process during connection to get a common wall clock for each client and the server to refer to, and then the clients can adjust their in-game clock appropriately to maintain themselves in the future slightly.

To support massive player counts and/or high ping servers, we also support the snapshot interpolation method as a server configuration setting.

Share this post


Link to post
Share on other sites

The Jitter Buffer section implies that state updates from the server arrive at the clients slightly before the client is due to present that frame. This means that he's running the server in the future / clients in the past (like counter strike) and not how I've described in my last post. 

The downside here is, in order for Sever updates packets for frame F to arrive just before frame F occurs for a client, the client clock has to be synced to at least ping/2 ms in the past relative to the server clock. This means that when the client sends user input to the server for frame G, the server will receive it at time G+ping. This will be fine for completely predictable actions, like moving a single FPS character -- there is latency, but the client can hide it by applying the inputs on their end at frame G -- this is incorrect / divergent from the server's version of events (because the server will apply the change at G+ping), but if no other interactions occur, the end result / final position of the character will be the same on both ends.  You only run into issues when your predictions are wrong because there's other things going on in the world, e.g. When two different FPS characters walk into each other, or you walk into a door/moving object being controlled on the server, or if two people throw physics objects at each other, there's some kind of timing/rhythm puzzle, etc, you'll notice ping ms of input latency. AFAIK, counter strike doesn't address that at all / it's just a tolerated downside. However, they do address the special case of weapon ray-tracing, by having the server rewind all player skeletons when receiving a shooting input, so that input lag of shooting is corrected for. The mostly correct predictions plus rewinding of weapon hit boxes means that it seems rock solid. 

Share this post


Link to post
Share on other sites
22 hours ago, Hodgman said:

Can you link to which of gaffer's work you're following?

I'm not sure if he does this (I'll have to re-read his work :)) but that's exactly what we do when using synchronized physics simulation in our racing game. It does add a lot of extra CPU load on the clients, especially at high ping...
In the "Counter Strike" model, the reverse happens -- the server pays the cost of constantly rewinding / fast-forwarding.

Also, just for clarity, the clients actually have to simulate / extrapolate to a point in time that's at least ping*0.5ms ahead of the server clock -- so that their input commands will arrive at the server just before the point in time when they should be applied to the simulation. We do a clock-synchronization process during connection to get a common wall clock for each client and the server to refer to, and then the clients can adjust their in-game clock appropriately to maintain themselves in the future slightly.

To support massive player counts and/or high ping servers, we also support the snapshot interpolation method as a server configuration setting.

The one quoted by lawnjelly. Thanks though, this kinda confirms my suspicion. I also do similar things as yours, that is, the client "tick" is continually adjusted so when it sends input at tick "T" it will arrive about at tick "T" on the server. Then the server sends snapshots to all client, and the client do physics rewinding. It's not quite perfect cause the server just sends states. Next step I'm gonna make the server to send inputs too. So far the client usually have to rewind 5-6 ticks, and capped at 16 ticks at max, which would make it jittery if there's high ping in the client.

It does add CPU Time though, but if it's the correct way of doing things then I'm okay. Kinda weird though since it feels like we're punishing the lagging client lol.

15 hours ago, lawnjelly said:

 

Share this post


Link to post
Share on other sites
21 hours ago, Bow_vernon said:

It does add CPU Time though, but if it's the correct way of doing things then I'm okay. Kinda weird though since it feels like we're punishing the lagging client lol.

So far the client usually have to rewind 5-6 ticks, and capped at 16 ticks at max, which would make it jittery if there's high ping in the client.

I don't know if it's the "correct" way to do things, I'm just making up game/engine architecture as I go along.  I'll let you know after I successfully launch a game using this model :D

For what it's worth though, Doom 3 used this model of client-side rewind and reprediction. There's a very in depth write up here: http://mrelusive.com/publications/papers/The-DOOM-III-Network-Architecture.pdf

As for the punishing ping thing... Yeah that's my main objection to this model at the moment. I'd love to support fairly high pings, as my racing game doesn't have collisions, so high ping shouldn't be as much of an issue. 

I haven't implemented these yet, but my plans are to automatically tweak some settings on the client when their ping is too high:

* increase the physics tick size during reprediction - e.g. Instead of doing 10x 60Hz ticks, run 5x 30Hz ticks to repredict. 

* spread reprediction cost over multiple frames. e.g. When the client gets a state update that's 10 frames old, they currently save their game state, load the servers state snapshot, run 10 ticks, then compare the new state against their saved one and add differences into the visual smoothing buffer. Instead, when getting that 10 frame old server snapshot, then could save their game state,load the servers state snapshot, run 5 ticks, save this as a new fake "server snapshot", load own their saved game state from before this prediction and advance to the next frame as if nothing happened. Next frame they take that half repredicted gamestate from the previous frame (which is now only 6 ticks out of date instead of 10) and pretend that it just arrived from the server. 

Share this post


Link to post
Share on other sites
Quote

increase the physics tick size during reprediction

This will guarantee that your physics simulations do not match up between client and server.

Unreal does something like this (frame rate dependent simulation) and thus doesn't have 1:1 positions on client/server.

The basic idea behind "zen of networked character physics" is to keep them together, though.

In general, though, assuming you don't resimulate your entire physics world, but only the player, simulating 10 steps should be very cheap. You may only need to do this if the difference at that past step between your old snapshot and the received checkpoint is greater than some small delta, although you make take the GGPO approach and just always re-simulate to show the player "as correct as possible" -- if it's cheap, might as well! Computers are fast.

Share this post


Link to post
Share on other sites
3 hours ago, hplus0603 said:

This will guarantee that your physics simulations do not match up between client and server.

They already don't match up before this :D (we use PhysX which is non-deterministic, and also sometimes user inputs from clients will arrive slightly too late to be applied at the exact frame that they wanted to apply them on) so the visual position of each player's vehicle is smoothed out to hide the corrections, while the physics state is constantly snapping with corrections from the server.

If I end up using a larger tick size for high ping players, they'll end up with more physics snapping = more weird visual smoothing... but the severity of these issues will depend on how bad their ping is. I think gamers kind of expect to have a worse network experience in fast paced when their ping is high, so it might be acceptable..?
Using the client-side reprediction model, the current behavior is that high-ping players end up with higher and higher CPU loads  / their framerate drops lower as their ping increases!! That's pretty awful, so I think having horrible physics quality at high ping is a better outcome than horrible framerate :| 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!