5 replies to this topic

[tom_mai78101]

Please assume me as a guy with a background in Java programming.

Is there a basic packet design in use for sending custom data, preferably game related information, to servers and/or clients? Any data you can think of is absolutely helpful. My purpose is to understand what a fundamental packet would look like for a packet sent from a game to some place, that allows multiplayers and stuffs. And what a packet does need as a prerequisite.

Does it involves implementing a Runnable? Does it require that it implements Serializable? Will a packet contains a few ArrayLists as member fields? I'm just looking for an easy, generic design. I know the question itself is vague, but that's my point. I just wanted to see what it looks like in the first place.

Thanks in advance.

Edited by tom_mai78101, 17 November 2012 - 12:22 PM.

[rip-off]

This is indeed a vague area, one that I think is impossible to answer definitively without context. A networked death match FPS game will have a completely different solution to something like a chess game. A browser game might be written in terms of HTTP requests, possibly encoded in something like Json or XML, while a desktop client might use a custom binary protocol. A MUD could be designed where the client is a dump terminal, with a simple line based textual protocol.

Have you looked at the Forum FAQ, in particular Q12.

One point I can make is that many games distinguish "packets" from "messages". A message might be something like "a new explosion was created at X,Y,Z" or "object PLAYER_4 has position P and heading H" or "actor PLAYER_4 has died". A packet will generally include all the messages that have occurred since the last packet was sent. A packet will often have a header which contains some metadata about the packet, for example it might contain information related to reliability, ordering, acknowledgement, ping timings and even flow control. The header complexity tends to be greater in a game that uses a lossy datagram style rather than a game that uses reliable streams. Time sensitive and action oriented games tend to use the former style, while turn based games tend towards the latter.

It is possible to design a "general" API, but there must be a few more constraints decided first. Raknet, for example, is a general game networking API but it places emphasis on low latency action oriented games (at least at the time I last used it).

Does it involves implementing a Runnable?

This is a completely implementation specific question. No, there is no necessity for a "packet" to be runnable. However, this might be part of a solution to move networking logic to a separate thread, if desired.

Does it require that it implements Serializable?

No. Using the built in Java serialization might be a great way to bootstrap your game, and for certain types of games (e.g. our aforementioned chess example) might be all you need to do. However, Java serialization has a number of draw backs. The biggest one, from a game programmer's point of view, is that the overhead tends to be very large. Another issue is handling protocol version changes - though this isn't an easy one to do in general.

Will a packet contains a few ArrayLists as member fields?

It is conceivable that someone would write a class "Packet" like that. However, a packet tends to be the lowest stage of the networking, at which point you are generally talking about arrays of bytes rather than higher level constructs. Some designs might not even model packets using a class - they might serialise messages to a byte buffer which is wrapped in a datagram or written to the stream.

[hplus0603]

Packets are just byte arrays. Serializable is generally terrible, because it adds all kind of reflection and type information that bloats your packets for no good reason. You already know what the bytes mean.

A typical game packet will look something like this, at the byte level (assuming UDP):

[a] [b] -- protocol version bytes / magic header
[c] -- cyclic sequence number of this packet
[d] [e] -- lowest two bytes of timestamp when this packet was sent
[f] -- last sequence number received by the other end
[g] [h] -- lowest two bytes of timestamp provided by the other end for the packet with id [g]
[i] [j] -- duration of processing between receiving packet [f] and sending out this packet [c]
[k] -- source of the packet (player id, or server)
[l] -- destination for the packet (player id, or server)
[m] -- number of messages in this packet

followed by, for each message:
[a] -- size of message (after size, channel, and type)
[b] -- logical channel for message
[c] -- type of message
[[a] bytes] -- payload according to [a]

finally followed by:
[x bytes] -- checksum, CRC, or signature of packet

Each message type [c] maps to a different layout of the packets, and should be interpreted differently.

There is also lots of variation.
Sometimes, the destination isn't included, because the destination is implicit in where you send the packet.
Sometimes, the timing information is not included, if calculating accurate round-trip-time or server-time-offset isn't needed.
Sometimes, there's not multiple logical channels in the stream, so the channel id isn't needed.
Sometimes, payloads bigger than 255 bytes per message is needed, and larger size fields are used.
Sometimes, integers are encoded using variable length (for example, if high bit is set, use the lowest 7 bits, shift left 7 bits, and then attach whatever comes after it, repeat until get a byte with high bit clear.)
Sometimes, more acknowledgement information is included, to support advanced forms of selective re-transmit, packet scheduling, etc.
Sometimes, the destination values are bigger, because you can target any particular object or entity within the system, and there are many of those.

There's a lot more variation, even within commonly accepted best practices. Compression, bit stuffing, implicit length by type, the list goes on. There are also games that don't use best practices, and send data in text form instead of binary, or send one packet per message, or do all kinds of other unnecessary things. However, if you start with this packet structure (again, assuming UDP,) then you're doing fine.

If you're using TCP, you have to additionally prefix each packet with a length field; typically 2 bytes (as you don't want to send more than 64kB of data at a time anyway.) You do not need a "magic" header/signature for TCP, as the protocol version stays the same for the duration of the connection. You may not be as concerned about timing with TCP, because it will deliver everything in order with larger amounts of jitter than you'll see with UDP. You do not need acknowledgement information for TCP, because it takes care of re-transmitting. The list goes on :-)

[tom_mai78101]

Thanks rip-off and hplus0603!

Have you looked at the Forum FAQ, in particular Q12.

Yes, but it narrowly answers a part of my question. At least, from that, I am able to understand what a network packet's content may want to have, depending on the game that uses the packet.

This is a completely implementation specific question. No, there is no necessity for a "packet" to be runnable. However, this might be part of a solution to move networking logic to a separate thread, if desired.

Oh I see. I tend to see them as together, but I don't know why it has to be together, until now. Moving network logic to a separate thread seems to be a very reasonable answer.

No. Using the built in Java serialization might be a great way to bootstrap your game, and for certain types of games (e.g. our aforementioned chess example) might be all you need to do. However, Java serialization has a number of draw backs. The biggest one, from a game programmer's point of view, is that the overhead tends to be very large. Another issue is handling protocol version changes - though this isn't an easy one to do in general.

Since I haven't had much experiences in Java serialization, before, I was thinking if a packet were to be thought as a binary file, wouldn't sending a binary file, in the order of a standard network packet, be easier to read/write? Now, this is my first taste of knowing that there would be drawbacks, something I never thought it would be that severe before. I'll keep my head up regarding this.

It is conceivable that someone would write a class "Packet" like that. However, a packet tends to be the lowest stage of the networking, at which point you are generally talking about arrays of bytes rather than higher level constructs. Some designs might not even model packets using a class - they might serialise messages to a byte buffer which is wrapped in a datagram or written to the stream.

I was worried that I have to write a few packets entirely in bytes. What a relief.

Edited by tom_mai78101, 18 November 2012 - 07:18 AM.

[hplus0603]

I was worried that I have to write a few packets entirely in bytes. What a relief.

I don't think that's a bad thing. You can model the contents of packets as structs/classes, and then actually generate the reading/writing to/from byte arrays as functions, perhaps on a general interface that each of your messages implement.

reading/writing fields of a message isn't that hard to do manually, and gives you great control over how your protocol grows.

class WhisperMessage extends Message {
  public int toPlayer;
  public string text;
  private static const int code = 10; // should not conflict with any other message class
  private static const registration = new MessageRegistration<WhisperMessage>(code); // for finding class on receipt
  public WhisperMessage(int p, string t) {
    super(code);
    toPlyaer = p;
    text = t;
  }
  protected void writeData(ByteStream strm) {
    strm.writeInt(toPlayer);
    strm.writeString(text);
  }
  protected void readData(ByteStream strm) {
    toPlayer = strm.readInt();
    text = strm.readString();
  }
};

Class Message would have public read/write functions that take care of also adding the type code and length fields to the output stream, as well as static functions to read message instances from a byte stream (you need to register the code with some dipatcher).

Also, please apologize if my Java is not quite syntactically correct; it's been several years since I last wrote any.

[Bubsy]

I was worried that I have to write a few packets entirely in bytes. What a relief.

I don't think that's a bad thing. You can model the contents of packets as structs/classes, and then actually generate the reading/writing to/from byte arrays as functions, perhaps on a general interface that each of your messages implement.

reading/writing fields of a message isn't that hard to do manually, and gives you great control over how your protocol grows.

class WhisperMessage extends Message {
  public int toPlayer;
  public string text;
  private static const int code = 10; // should not conflict with any other message class
  private static const registration = new MessageRegistration<WhisperMessage>(code); // for finding class on receipt
  public WhisperMessage(int p, string t) {
	super(code);
	toPlyaer = p;
	text = t;
  }
  protected void writeData(ByteStream strm) {
	strm.writeInt(toPlayer);
	strm.writeString(text);
  }
  protected void readData(ByteStream strm) {
	toPlayer = strm.readInt();
	text = strm.readString();
  }
};

Class Message would have public read/write functions that take care of also adding the type code and length fields to the output stream, as well as static functions to read message instances from a byte stream (you need to register the code with some dipatcher).

Also, please apologize if my Java is not quite syntactically correct; it's been several years since I last wrote any.

Better yet, make the Message class realize the Externalizable interface, which provide those public read and write methods and is seamless pluggable to Java's default serialization framework.

http://docs.oracle.com/javase/7/docs/api/java/io/Externalizable.html