I am curious as to how effective my current method of formatting the bytes in packets for multiplayer games is. I have made basic multiplayer games in the past (multiple users in an environment who can move from a(x,y) to b(x,y) with A* pathfinding on a grid).

I have always formatted my information as such:
First 2 bytes describe how many bytes following are part of the command (unsigned short)
Second 2 bytes describe the op-code for this particular command
Rest of the command's bytes are formatted in the way understood by the particular op-code to perform the command, and are extracted and typecasted appropriately

The networking code can then seperate the commands from the in buffer in order because it is aware of how many bytes each takes up

Multiple commands can be put into one packet, provided the packet is transmitted cleanly, there were no issues with my technique (I was using TCP just to test it out). Of course, the operation doesn't require that it works like a command, for example: If I was sending the player's location in a fps style game it would have 2 bytes for the length of the "command" then 2 bytes for the op-code followed by 4 bytes to describe a player's location at that time. In the same tick on the client, a command would be at the start of the packet to describe the time signature of that particular tick. Data could be collected over a period of 100ms to interpolate the movement without predicting the future (causing a small lag, but it works for the source engine so it must be good enough for me), or it could predict one tick into the future after the 100ms history of locations.

I was hard coding all the op-codes in a switch statement to keep it robust, but I feel I need to do something more adaptable if I intend on making a proper networked indie game.

I understand I could use something like raknet to do all this (and more) for me, but I want to learn everything from the low level to keep it as efficient as possible and so that I have a much greater depth of understanding. Replicating objects and values between devices sounds extremely bloatworthy and hackable - especially if I am unaware of the specifics underneath the library.

Are there any good articles or white papers regarding what I am trying to do? I thought up my method myself because I was unable to find a proper example of how to do it correctly. I will also have to learn how to synchronise threads and communicate between them properly.

/wall of text

Quote:Original post by smasherprog
I was hoping that he could think it out himself instead of just being an answer machine. That is why I asked about those two questions. Since you already suggested it; for my networking, I use your suggestion: lead with the op-code, then pass the rest of the packet to a function that knows how to deal with the rest of the data.

The advantage of having size first is that it allows processing of unknown commands (IFF and similar). Aside from trivial error checking it also allows robust transition between versions. Another benefit is that it works over TCP with no modification (handles buffering implicitly)

But perhaps more importantly - diagnostic and debugging. By having explicit payload sizes, the inspection tool can be dumb yet still separate the content, perhaps even log it. With fixed per-message header, this tool can classify and dump from raw stream without knowing the contents - or without performing possibly complex session/protocol/version correlation, such as when diagnosing client upgrade issues or if multi-version clients are allowed in first place.

Quote:To me, this leads to simpler code: just need a leading op-code (fixed size of 2 bytes is more than adequate), then the remaining size is known to the specific function.

The downsides are:
- there is only one op-code/message per UDP packet or broken message permanently breaks the stream with TCP
- if protocol changes in any way, or an old-version client connects, it will not be able to interpret unknown or changed messages or even detect them. Without fancy serialization it could also lead to annoying issues, such as data order change means what used to be Y is not string length

Considering minimal 42 byte overhead of TCP, extra size bytes are fairly good investment since they buy forward and backward compatibility and robustness.


Usually, when connection is established, version parameter would be sent as well. While this could be used to synchronize peers to proper version, it doesn't alleviate the other problems.



And for anyone who wants to be fancy, there's always possibility of variable length encoding of payload size (see UTF8 for an example) or perhaps mangling bits of opcode in there as well, or perhaps doing some kind of dynamic huffman scheme to generate opcodes on the fly (might help with replay attacks).


But considering all the noise that is going on over the wires these days in form of escaped JSON encoded via MIME over HTTP, the two bytes won't really show up anywhere.

wow, I didn't realise I was doing it so similarly to how others do. I only recently discovered the struct passing method (havn't used C++ forever). When I first programmed a networked game I had the client in game maker and the server in a php command line daemon (due to the game maker sockets extension pulling data into a string I had to avoid null bytes and special ascii characters in the packets by having no byte value less than 32! But that was years ago and I don't need game maker anymore :P).

I am confident that my technique will suite my needs from now on, I was expecting to be miles off. I think in actual fact I have been sending the command length first - I typed up my thread without my old code for reference and I wasn't sure how I had done it. To tell the truth though, I didn't spot that either way was different from the other (code or length first).

Thanks for the help, now I need to delve into winsock properly - I should make a sockets wrapper first though so I can develop the server for linux and windows platforms with the same socket functions.

Now I need to worry about synchronising and controlling a stable tick rate to synchronise the nodes with. And things such as late-joining and application hangs.

Not to mention the multithreaded requirements of a proper networking system. Luckily though, I only need something that will work over a LAN first (TCP should be ok) then I can take a broad look at my design and take what I have learned to design a new system that will be functional over the internet. I know my first attempt will be flawed and I am expecting that.

Here is the contents of a notepad document that I have been putting my thoughts into:

Server-Client synchronisation.

- Tick commands will describe what tick the commands following them occur at, eg:
[tick 1449][player location update][player reload][player location update][tick 1450][player location update][player location update][player shoot]
This prevents a tick value needing to be sent with every command. The networking code can split
the packets up into tick information. If an output packet being prepared becomes too big for the MTU, the
previous tick still stands as the tick for commands at the start of the next packet. What implications
will this have? Provided that this only happens when the connection is unsatisfactory, it could be allowed
to have an effect on gameplay - ie, dropped commands (the tick will already be dealt with on the receiving
end and the information is now out of date). Therefore, certain pieces of information would not need to be
sent at that point - ie, values which are instantly out of date if they are updated on the intended tick for
the packet. Should packets always have the true tick at the start, followed by commands - even a command
dictating the previous tick followed by out of date information from that tick? When the packet is prepared
for sending - anything from the previous tick which is altered in the current tick should be stripped out
and not even sent across the network.

- The simulation (at least the networked component) runs at a specific tick rate.

- Interpolated values (such as moving object locations and angles) are collected over a short period
of time (such as 1/10 of a second, or a number of ticks) before the object is interpolated along the
given points. This creates a minimum lag but all movement/turning should appear smooth.

- All devices (server and client) count the tick by themselves but would ideally be on the same tick
at any given point in real time, though expected lag will mean this isn't fully possible. I am not
sure if the tick will dictate any other things within the game such as a reference point for other
timing - but it is what dictates the network and game logic. The PhyreEngine may not have been
designed with real-time interactive multiplayer in mind.

- The server is authoritative regarding the correct tick that clients should be on.

- A client can drift a few ticks ahead or behind the server (values can be set in configuration).

- If a client's tick drifts too far away from the server's, the client will be "bungee roped" forwards
or backwards into synchronisation - this requires some investigation.

- A late joining client will have its tick adjusted over a short period of time after connecting
when enough information is discovered (lag compensation) to decide what the correct tick should be.
clients could count from 0 and use an offset, but this is not any different from resyncing them.

- Ticking may be calculated on the networking thread or a separate tick calculating thread, this
must be calculated as accurately as possible. Due to scheduler and multithreaded application
quirks and low level issues, only some tick rates will be possible to maintain - this requires some
degree of investigation because the method must be stable and accurate for as long as possible.

Quote:Original post by Bozebo
Thanks for the help, now I need to delve into winsock properly - I should make a sockets wrapper first though so I can develop the server for linux and windows platforms with the same socket functions.

While I agree with you about wanting to develop your own server infrastructure instead of using something like RakNet, I think you would be better off using an existing socket wrapper like boost::asio. It handles a lot for you, and gives you cross-platform code, though be warned that you will probably have to do a lot of research before you find examples of asio that do exactly what you want.

[quote "Bozebo&quot;]but I want to learn everything from the low level to keep it as efficient as possible and so that I have a much greater depth of understanding. <br /> As he said. He doesn&#39;t want to use an external lib. <br /> <br /> There is nothing wrong with going low level. <img src='http://public.gamedev.net/public/style_emoticons/<#EMO_DIR#>/smile.gif' class='bbc_emoticon' alt=':)' />