Hello,
We're trying to write a network synchronization for a physics based 2D shooter (in Java)(website). Its our second attempt but we feel it won't work again.

Our first attempt was:
1) Server builds sync packages which contain position + rotation updates all as byte arrays
This usually ended in a mess with Java's ByteBuffers but we got it working for some basic parts
2) Client receives sync packages and decodes the byte array and applies updates
Like for 1) it was a mess with bytes

3) Important events like "add physics body" or "shoot gun" had to be sent to server and acknowledged/accepted before client could add a bullet
Bullets are physic bodies too. Adding physic bodies (same fore loading the level) also was a command witch was nearly 100% synced over network.

But all together never worked well over network and after a while every thing starts to slow down on the client side or events were applied seconds after they were physically sent.

The protocol was written using a UDP socket. I implemented functions that could guarantee package transfer and functions that did not guarantee but gave a good speed. The unsecure functions were used for 1) and 2) and the secure functions were used for 3)

Our main question is: Is there a much easier way to implement a working synchronization over the network? Are there any libraries that do exactly what I need here?
Is Java's serialization API something that could help?

Thanks for any comment!

Hi,

I am the other developer on this project and I'd like to add something:

I've read the articles on gafferongames.com (which were mostly on the topic of physics-engine synchronization) and some other articles which were really quite helpful but we both wondered why there never was any real discussion about how the data that is sent over network is created in the first place.
We just found it to be a lot of effort to write the code that creates Strings/bytes for every object in the code and additional code that deciphers it and converts it into real data and even more "annoying" to always keep that code up with the current state of the game. We find it kind of hard to imagine that no one else has come across this problem, but we couldn't seem to find any real articles or presentations about that topic.

Thanks in advance,

Tehforsch

Our first attempt was:
1) Server builds sync packages which contain position + rotation updates all as byte arrays
This usually ended in a mess with Java's ByteBuffers but we got it working for some basic parts
2) Client receives sync packages and decodes the byte array and applies updates
Like for 1) it was a mess with bytes[/quote]

Hmm, Java?

implement Serializable, add GUID, done. Just keep classes sent over network simple to avoid deep copies. They should also be stateless (as in, have no class invariants) to avoid having to write custom serializers.

If classes are kept simple (primitives, lists, maps, ...) and POJOs (no complex construction, dependencies), there's no extra work. They should be kept that way anyway for sake of security.

To pack/unpack, just write instances of Object to ObjectOutputStream, ObjectInputStream to reverse. Managing GUIDs may be a bit annoying, but I don't recall it as ever being a problem. And today one might just hook a post-commit hook on git to generate those and keep versions up to date. They can be frozen once a milestone release occurs.

If you google around github, there are some other encoders which offer better buffer representation (smaller, less metadata). For security, you can later filter the type of instances you're allowed to send/receive. Could be done through reflection.

Raw Serialization is pretty simplistic and has some gotchas, but if staying away from JEE and Bean horrors it is quite straightforward, even though .net became vastly superior in this area.

One example about how to go about this:class SyncUpdate implements Serializable {
List<Vector2D> positionUpdates;
List<Vector2D> velocityUpdates;
List<String> otherUpdates;
};Your code now just adds stuff to these lists, then sends instance of SyncUpdate. Other end reads it and that's it. Your code only ever works with Java. It should be pretty simple to go from here.

[color="#1C2837"]What you want is a library for marshaling/demarshaling. There are many different approaches you can take.[color="#1C2837"]What you need to do is three things:

1) Make sure you can pack messages into packets such that the remove end can unpack the messages -- typically, this is a byte or two of length information per message within the packet.
2) Make sure that you can tag each message so that the remote end knows what particular code should receive that message -- typically, this is a byte or two of object ID, where "object" may be a player avatar, or a subsystem (such as the ammunition manager, or whatever)
3) Make sure that you tag each message such that the receiving code knows what to do with it -- typically, this is a type code of some sort.

One way of approaching this is to build marshaling on top of reflection. Unfortunately, C++ does not have reflection, so you'll have to add it yourself.
A few articles on this subject:
http://www.enchanted.../cpp-reflection
https://github.com/j...-and-Properties (code for my article in Game Engine Gems 2 about this)

[color="#1C2837"]http://www.enchanted...2-introspection[/quote]


Hi, thanks for your reply.
We were doing something similar to this in our previous version (luckily we are using Java, so serialization and reflection are both available, thanks for the links anyways!).

We had a working version were the initial state of the game could be loaded on the client, and objects could be added by the server.
The main problem I had was how to write the code that synchronizes the changes in game objects for all clients. We have game-object structures that contain a list of physical and graphical properties as well as game logic code. We were able to get the parts of the engine (physics, graphics) across the network but we never found a way of writing nice code to synchronize the varying game logic of each object. But it seems like there is no way around writing separate code for each object? Most objects are just crates or bullets anyways and don't really need any synchronization but there are others like the player figure itself which consist of a lot of attributes and whenever we wrote new code for the player we had to extend the synchronization code aswell, which was kind of annoying.

Hmm, Java?

implement Serializable, add GUID, done. Just keep classes sent over network simple to avoid deep copies. They should also be stateless (as in, have no class invariants) to avoid having to write custom serializers.

If classes are kept simple (primitives, lists, maps, ...) and POJOs (no complex construction, dependencies), there's no extra work. They should be kept that way anyway for sake of security.

To pack/unpack, just write instances of Object to ObjectOutputStream, ObjectInputStream to reverse. Managing GUIDs may be a bit annoying, but I don't recall it as ever being a problem. And today one might just hook a post-commit hook on git to generate those and keep versions up to date. They can be frozen once a milestone release occurs.

If you google around github, there are some other encoders which offer better buffer representation (smaller, less metadata). For security, you can later filter the type of instances you're allowed to send/receive. Could be done through reflection.

Raw Serialization is pretty simplistic and has some gotchas, but if staying away from JEE and Bean horrors it is quite straightforward, even though .net became vastly superior in this area.

One example about how to go about this:class SyncUpdate implements Serializable {
List<Vector2D> positionUpdates;
List<Vector2D> velocityUpdates;
List<String> otherUpdates;
};Your code now just adds stuff to these lists, then sends instance of SyncUpdate. Other end reads it and that's it. Your code only ever works with Java. It should be pretty simple to go from here.

This is what we did in our previous version. What I had most problems with is - in your code example - the "otherUpdates" part. how exactly do games with a lot of code create those? Is there any way around writing individual sections for each object that you create?

Thanks for your replies,

Tehforsch

Have an example of how it's currently done?

Have an example of how it's currently done?

We have been rewriting the entire game engine completely so this is not very accurate but this is some sample code from the last version of the game:

Some of the code for the server:

private Vector<Package> createSyncPackages() {
Vector<Package> packages = new Vector<Package>();
if(this.world == null) return packages;
for (int i = 0; i < world.numBodies; i++){
Body b = world.getBody(i);
Package p;
if (!bodiesOnClient.contains(b.id)){
p = new Package(Name.ADDB, Savegame.create(b));
bodiesOnClient.add(b.id);
}
else{ // Synchronize body
if (b.hasToBeSynced())
p = new Package(Name.SYNCB, b.createSync());
else continue;
}
packages.add(p);
}
for (int i = 0; i < world.numJoints; i++){
Joint j = world.getJoint(i);
Package p;
if (!jointsOnClient.contains(j.id)){
p = new Package(Name.ADDJ, Savegame.create(j));
packages.add(p);
jointsOnClient.add(j.id);
}
else if (j.hasToBeSynced()){
p = new Package(Name.SYNCJ, j.createSync());
packages.add(p);
}
}
for (Integer k : bodiesOnClient){
if (!world.containsBody(k)){
Package p = new Package(Name.REMB, Package.getBytes(k));
packages.add(p);
}
}
for (Integer k : jointsOnClient){
if (!world.containsJoint(k)){
Package p = new Package(Name.REMJ, Package.getBytes(k));
packages.add(p);
}
}
setClientCam(packages);
return packages;
}

And some of the code for the client:

private void handleReceivedPackage(Vector<Package> packages, Package r) {
Name name = r.getName();
byte[] data = r.getData();
switch (name) {
case ADDB:
Body b = Savegame.loadBody(data);
if (!world.containsBody(b))
world.addBody(b);
packages.add(new Package(Name.RECADDB, Package.getBytes(b.id)));
break;

case ADDJ:
Joint j = Savegame.loadJoint(data);
if (!world.containsJoint(j)){
world.addJoint(j);
}
packages.add(new Package(Name.RECADDJ, Package.getBytes(j.id)));
break;

case REMB:
int id = Package.getInt(data, 0);
if (world.containsBody(id))
world.remBody(world.getBodyById(id));
packages.add(new Package(Name.RECREMB, Package.getBytes(id)));
break;

case REMJ:
id = Package.getInt(data, 0);
if (world.containsJoint(id))
world.remJoint(world.getJointById(id));
packages.add(new Package(Name.RECREMJ, Package.getBytes(id)));
break;

case INIT:
World w = Savegame.loadWorld(data);
world = w;
packages.add(new Package(Name.RECINIT, new byte[0]));
break;

case SYNCB:
id = Package.getInt(data, 0);
world.getBodyById(id).sync(data);
break;

case SYNCJ:
id = Package.getInt(data, 0);
world.getJointById(id).sync(data);
break;

case SETCAM:
id = Package.getInt(data, 0);
centeredBody = world.getBodyById(id);
packages.add(new Package(Name.RECSETCAM, Package.getBytes(id)));
break;

}

}


Note that the REC... packages are sent from the client to the server to acknowledge that the client received the corresponding package. If the REC... did not arrive for some amount of time the package would have been sent again.

The server is constantly keeping track of the objects that the client has actually received (by their unique id) and whenenever an id is missing, an ADD... package is sent, if an id is on the client but shouldn't be (the object has been removed) a REM... package is sent.

This code actually worked quite well, but I have to add that we didn't actually synchronize any of the game code. The client was just a standalone physics/graphics engine and would only display and simulate the world, not know what was happening. That way we only had to synchronize the physical world and some graphical attributes like the camera position etc. This was more or less just a test to see if it was possible to have the game logics running only on the server since that would have saved a lot of effort. We had a version where the game was send to the client, but except for initializing the objects on the client we were never really able to make it work.

Thanks,

Tehforsch