Sign in to follow this  

Physics synchronization over network

This topic is 2604 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

We have a server that has physics processing module (rigid bodies). Need to send minimal data to clients to see the same scene.

As I understand, sending positions and orientations is not good because if body moves then client gets about 40 bytes per frame, so maybe 400 bytes per second. It's just for one object. But if there's a hundred?

So good solution as I understand is to send position, orientation, linear and angular velocities. So if body has linear movement, nothing is being sent. The problem is when to send? Maybe the solution is to emulate client on server - to store those data and update like pos = pos + vel * dt, quat = ... and check the difference between real position and orientation, and between this client emulation. So if difference is more than 0.1 then update client (send all new data). This is bad with quaternions, because they are not linear and the difference |x1 - x2| + |y1 - y2| + |z1 - z2| + |w1 - w2| will be different on different angles.

So what to do?

Share this post


Link to post
Share on other sites
How critical are these objects? Do all of these hundreds of objects actually impact game play? (Can players take cover behind them, run on them, etc). If not, you can just have the client guess and throw objects around based on whatever data it last got from the server. If they are, you're going to have to come up with some kind of priority scheme. You probably won't be able to send data for hundreds of objects over the network at once. What you can do is calculate the closest objects, objects the player is looking at, etc, and give them a higher priority, send their state (position, orientation, velocity) to the clients, and have the clients run physics until they get the next state. Low priority objects can be continuously upped in priority until they are finally sent and everything in the world is updated.

Share this post


Link to post
Share on other sites
I didn't mean that I have millions of objects. Just hundred. If I send 40 bytes * 100 objects * 10 times per second I get 4 kb per second, but it's too much for falling and rolling balls for example. If all of them are moving on the flat ground elements, then we can send their data every 5 seconds for example, because client just makes x = x + v * dt.

Share this post


Link to post
Share on other sites
Yes, it is very common to do "baselining" where you send a compressed snapshot of the state of the object for certain times (say, once every N seconds), and then only send "change" events, like when the server detects a collision between objects, in the meanwhile.
You also need to send player inputs in a constant stream, so that player entities keep updating the simulation.

Because of the transmission latency between clients and the server, there are two main problems you'll need so solve here:
1) players colliding with other players -- you don't have perfect knowledge, so the players will see different things; you have to reconcile this
2) multiple players wanting to collide/interact with the same object. For example, picking up a health pack from the ground -- who got there first?

Share this post


Link to post
Share on other sites
In the past I've had success with slerping quaternions or using a data filter with upper and lower boundary checks with wrap around. Like this: http://www.replicanet.com/assets/docs/Main/html/classRNReplicaNet_1_1DataBlock__Predict__Float.html#da3fce1d4164fc4f075257b1406ee3eb

The quaternion approach was especially good with networked physics because rotations could use deltas.

Share this post


Link to post
Share on other sites
hplus0603, you always describe small manual how to make a game (something like this). I don't have problems with synchronization of object getting. I just have physics server, so if player-A-packet-for-getting-object will be received first than player-B-bla-bla-bla then server lets player A get this object.

And yes, I know that quaternions are good for rotation, because without them we won't get right processing. But what about networking? Because physics engine has it's own updating pipeline, own stepsize (ofcourse it's fixed and about 0.2 seconds). But client has about 50 frames per second, so something like q_new = 1/2 * q_old * w is not equal for different step sizes. And if we process another formulae with quaternion exponents (that's equal for different step sizes), we get not-synchronized physics on client and on server.

I thought about getting the difference of quaternions (like q_diff = q_new / q_old) on every server step and then just interpolate on client, but rotations bigger than 180 degrees per server step are not right. Maybe it's better than not-synchronized physics on client and server that makes server send more packets on constant rotation...

Share this post


Link to post
Share on other sites
I don't think that will work as described. You need more smarts to make a distributed rigid body simulation works alright (this is why most multiplayer games have very little rigid body physics!)

First, the server needs to have a short time step (ideally, the same as clients), to avoid tunneling, and make simulation consistent.

Second, you want to assume that all players will simulate the same objects the same way most of the time -- the only case that can change that is player action. Thus, you want to keep track of which objects are being affected by players at what time steps, and send information about those objects.

Third, you want to snap the physics objects to the locations you get from the server, as soon as you get them (and then simulate them forward again to whatever time step you actually display to the client). The visual snap should be hidden in rendering, not in simulation; for example by calculating a delta error and lerping that error (both for angular and linear position) to 0 over a few frames of rendering.

Finally, you want to cycle through all your simulated bodies, and send a checkpoint for each body once in a while to all players, so that a body that goes out of sync will be put into sync again.

Share this post


Link to post
Share on other sites
First, client has about 100 "interesting" objects. Server has millions of objects, some of them are inactive. Client just renders good quality frames (shaders, etc) and server manages all the objects, so as I described I just need to optimize sending of physics data, not managing interesting objects, not managing players synchronization in their interaction. So server steps are too big, and client makes about 50 frames per second, on some machines I get 200 frames. Server cannot simulate physics on that speed...

Problem is in several words. Server sends physics data, not much, not too often, nothing else. I talk just about minimizing of physics data, that's optimized already (nobody gets information about all the objects, it's already divided to interesting collections).

So just data about position and orientations and their change so that client can represent object update in different frame rates, if object is constantly rotating, then we don't need to send data on every frame.

It's just like movie compression. We have film resolution about 1024x768, but it's not being saved as 2359296 bytes per frame. Some regions are constant, some regions are linearly moving. It's like my problem. Most of advices here are just like "please lower the resolution and you'll be happy" or "do not use 24-bit color" or "make 1 frame per second". So like I already have the task with high-quality videos, 24-bit color, high-speed - it's just an example.

Share this post


Link to post
Share on other sites
The clients should have the same physics code as the server. Think of the clients like DirectX States are done where the GPU continues using the last State until it is changed. Send the player the initial velocities, pos, etc of the object and then let the clients figure out most of the res while sending sync updates on some time delay that is the longest time you can delay without screwing up what normal players see on screen. These numbers are all relative to your simulation, and will need to be tested to find.

*edit*

Just read your above post and it seems like you already know the answer.

Share this post


Link to post
Share on other sites
Yes, 90% of answer I know is that we send position, orientation, their changes, and same delta processing on server and client. But quaternions are different from positions, where you can just process x_new = x_old + vel * dt, and vel * 0.5 seconds + vel * 0.5 seconds = vel * 1 second.

Share this post


Link to post
Share on other sites
Quote:
Server cannot simulate physics on that speed


You should not lock graphics simulation speed to physics simulation speed.
You should use fixed time steps, and render frames of graphics with whatever time is left (which may be more or less than the physics rate).
This means graphics needs to know how to interpolate between physics frames. That's good, because graphics also needs to interpolate to hide "snaps" in prediction.
Basically, you should use the client simulation as the prediction/extrapolation for each object.

That way, you get position, orientation, and even bounces predicted for free, so all you need to send is events where the client simulation would diverge from the server (such as colliding with an object that's invisible to the client). Plus a slow round-robin baseline to heal whatever objects might not have gotten caught by the event sending.

I think that what you are asking is "how can I make each physics frame smaller to send" and/or "how can I use interpolation to interpolate physics objects," when your end goal is "how can I consume less network bandwidth."
What I'm answering is "you don't want to interpolate physics objects; you want to run the simulation; you don't want to send lots of small physics frames; you want to send *no* physics frames most of the time."

Share this post


Link to post
Share on other sites
Yes, I understand that client just interpolates in own timesteps that not equal to server steps. How to send less data and make good result - how to make this interpolation.

In linear velocity and position it's easy - we can send position correction and velocity and client just moves object to that position and update it like pos_new = pos_old + vel * dt until next correction. Server makes the same processing and if the difference of this pos_new and position after REAL physics processing are very different, then send correction. Goal is that client processes 50 frames per second and server does 4 frames and pos_new = pos_old + vel / 4 4 times is equal to pos_new = pos_old + vel / 50 50 times so it's synchronized.

Rotations are not so easy. They use quaternions and we can solve as:
1. Sending quaternion correction and angular velocity. But there is 2 formulas, q_new = 1/2 * q * w * dt which is not equal for different time steps and we have some exponential formulae, but physics engines don't use it, so in any case we get not-synchronized physics: or server-client, or server_processing-physics_processing.

2. Send timesteps and calculate quaternion difference (this_step and last_step). Then we loose rotations more than 180 degrees per step.

Share this post


Link to post
Share on other sites
Quote:
Sending quaternion correction and angular velocity. But there is 2 formulas, q_new = 1/2 * q * w * dt which is not equal for different time steps and we have some exponential formulae, but physics engines don't use it, so in any case we get not-synchronized physics: or server-client, or server_processing-physics_processing.
Not sure what you're saying there, but applying a constant angular velocity isn't difficult, even with varying timesteps; just multiply the angular velocity vector by the timestep, interpret its magnitude as rotation angle, build the quaternion, multiply by the previous quaternion. The exponential map stuff going on there is all implicit.

Oh, and a minor quibble: The nature of quaternions (double cover of SO(3)) means that they can unambiguously represent rotations of up to 360 degrees per step, not just 180 degrees.

Share this post


Link to post
Share on other sites
Quote:
Original post by Sneftel
they can unambiguously represent rotations of up to 360 degrees per step, not just 180 degrees.


Imagine rotation 180 degrees per step. So we inform client "rotate this to 180 degrees", but interpolation can go from 0 to 180 or from 0 to -180... It's like in modelling programs when you rotate object to 181 degrees, it transforms to -179.

Share this post


Link to post
Share on other sites
Quote:
Original post by Anton Vatchenko
Quote:
Original post by Sneftel
they can unambiguously represent rotations of up to 360 degrees per step, not just 180 degrees.


Imagine rotation 180 degrees per step. So we inform client "rotate this to 180 degrees", but interpolation can go from 0 to 180 or from 0 to -180... It's like in modelling programs when you rotate object to 181 degrees, it transforms to -179.
Nope! With a quaternion, if you rotate 180 degrees, and then rotate another 179 degrees in the same direction, the result is a rotation of 359 degrees, which is numerically distinct from a rotation of -1 degrees. They're unlike other rotation representations in this respect.

Share this post


Link to post
Share on other sites
Try to find some online calculator and convert rotation to 359 degrees and to -1 degree. Result will be the same. It means that if we send in quaternion rotation to 181 degrees, then in quaternion style we get rotation to -179, so we cannot use more than 180 degree rotation per step.

But the problem is another - how to synchronize physics over network. In details... Should I send quaternion orientation and rotation speed and get unsynchronized server real physics and server easy-physics-calculator (that emulates client updater), or send some difference amount, but loose rotation more than 180 degrees per step, or other solutions?

Share this post


Link to post
Share on other sites
Here's an other approach: binary delta compression.

1. Try to reduce the obviously data (i.e. use only x,y and recalculate z when working with normals).
2. Try to compress all left data to i.e. fix point number of a certain bit width(i.e. map a float value which will be always between 0.0-1.0 to 0-255 or 0-4096, try to check which precision is enough for rendering).
3. Define one fix data block for each physical object inside your focus considering 1. and 2.
4. Defnine a memory block which could hold up to 100 datablocks.


struct {
int pos_x : 22;
int pos_y : 22;
int pos_w : 22;
int quat_x: 10;
int quat_y: 10; //note calculate z from x,y
int quat_w: 10;
} OrientationDataBlock;

OrientationDataBlock clientDataBlock[100];



Assign all visible objects to an according datablock. Clear a datablock if it is not necessary any longer. Don't shift datablocks, just "activate/deactivate" them.

Well, now you can work with the binary data of this client array of data blocks. Send the whole data block once at the start, after that remember always the last send data array. Now you need only to send the delta (=previous_send_data_block XOR current_data_block), this will contain a 1 bit only at bit-positions which has been changed. So, *hopefully* most of the time there are more 0 than 1 which can be easily compressed with a RLE compression. Send this compressed data block to the client and uncompress it there.

Improvement: when only few data blocks are changing in a single frame, try to "defragmentate" your data block by swapping used blocks from the end of the array to the front (unused blocks). This way you need only to send the data between the first and last used block.

Share this post


Link to post
Share on other sites
I'll repeat my sample with video compression. You advice something like "convert bright frame to quantized 256 colors palette". If I send data of hundred objects on every step, I get about 40 kb per second. Yes, if I use your advice, I'll get about 20 kb per seconds, but all of my possible solutions usually use 500 bytes per seconds, with no 8-bit positions, rotations etc.

Share this post


Link to post
Share on other sites
Anton: If you're saying the client will not run a physics engine, then you will not be able to properly predict the movement of objects that intersect terrain or other objects.

Dead reckoning linear and angular orientation based on snapshots is an old technique. Sending additional snapshots only when there is a given amount of delta between the extrapolated and real positions is also an old technique. This is well defined and well described in available distributed simulation literature -- for example, the DIS protocol (IEEE 1278) defines several modes of dead reckoning and delta compression for snapshots.

Regarding the rotation aliasing problem, there really is no problem. If you send rotational velocity as a denormal quaternion (really, an axis with a length), then the rotation across any particular time period is well defined no matter how long or short the time period is. If you sample it at long enough intervals, you will get sampling aliasing, where the object may appear to rotate in the other direction, or even stand still -- that's a sampling problem, not a representation problem.

So, if you want to keep this thread going, I suggest that you formulate very clearly what the point of your thread is.

1) Are you trying to select between a number of different algorithms? If so, you'll need to show specific numbers that relate to your problem at hand, for anyone to be able to give a good answer.

2) Are you trying to get pointers to mechanisms you can use to solve a particular problem of dead reckoning? If so, I suggest reading the reference linked in this post.

3) Are you trying to have another question answered? If so, you'll need to be more precise and concise in what the question is.

That being said, if the question is "what's the best way of minimizing bandwidth use when getting clients to view some subset of a server-side physical simulation," then the answer really is "also run (parts of) the simulation on the client." If you don't want to do that, then you don't want to use the most bandwidth efficient mechanism, it's as simple as that.

Share this post


Link to post
Share on other sites
Quote:
Original post by Anton Vatchenko
Try to find some online calculator and convert rotation to 359 degrees and to -1 degree.

Formula for axis-angle to quaternion:

Q(N,θ) = <sin(θ/2)*x, sin(θ/2)*y, sin(θ/2)*z, cos(θ/2)>.

For ((1,0,0), 359): <0.999, 0, 0, 0.008>
For ((1,0,0), -1): <0.999, 0, 0, -0.008>

Note the negative sign. The two are NOT the same. You can make them the same by negating any quaternion with a negative W component, and this is commonly done (google "quaternion neighborhood"), but as shown above, the two are distinct from each other otherwise.

(It's possible that whatever online calculator you used changed the quaternion neighborhood automatically, and that that's what's making you confused. Look over the actual equations for quaternions as rotations: It's the division of the angle by 2 that has this effect.)

Share this post


Link to post
Share on other sites

This topic is 2604 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Guest
This topic is now closed to further replies.
Sign in to follow this