• Announcements

    • khawk

      Download the Game Design and Indie Game Marketing Freebook   07/19/17

      GameDev.net and CRC Press have teamed up to bring a free ebook of content curated from top titles published by CRC Press. The freebook, Practices of Game Design & Indie Game Marketing, includes chapters from The Art of Game Design: A Book of Lenses, A Practical Guide to Indie Game Marketing, and An Architectural Approach to Level Design. The GameDev.net FreeBook is relevant to game designers, developers, and those interested in learning more about the challenges in game development. We know game development can be a tough discipline and business, so we picked several chapters from CRC Press titles that we thought would be of interest to you, the GameDev.net audience, in your journey to design, develop, and market your next game. The free ebook is available through CRC Press by clicking here. The Curated Books The Art of Game Design: A Book of Lenses, Second Edition, by Jesse Schell Presents 100+ sets of questions, or different lenses, for viewing a game’s design, encompassing diverse fields such as psychology, architecture, music, film, software engineering, theme park design, mathematics, anthropology, and more. Written by one of the world's top game designers, this book describes the deepest and most fundamental principles of game design, demonstrating how tactics used in board, card, and athletic games also work in video games. It provides practical instruction on creating world-class games that will be played again and again. View it here. A Practical Guide to Indie Game Marketing, by Joel Dreskin Marketing is an essential but too frequently overlooked or minimized component of the release plan for indie games. A Practical Guide to Indie Game Marketing provides you with the tools needed to build visibility and sell your indie games. With special focus on those developers with small budgets and limited staff and resources, this book is packed with tangible recommendations and techniques that you can put to use immediately. As a seasoned professional of the indie game arena, author Joel Dreskin gives you insight into practical, real-world experiences of marketing numerous successful games and also provides stories of the failures. View it here. An Architectural Approach to Level Design This is one of the first books to integrate architectural and spatial design theory with the field of level design. The book presents architectural techniques and theories for level designers to use in their own work. It connects architecture and level design in different ways that address the practical elements of how designers construct space and the experiential elements of how and why humans interact with this space. Throughout the text, readers learn skills for spatial layout, evoking emotion through gamespaces, and creating better levels through architectural theory. View it here. Learn more and download the ebook by clicking here. Did you know? GameDev.net and CRC Press also recently teamed up to bring GDNet+ Members up to a 20% discount on all CRC Press books. Learn more about this and other benefits here.
Sign in to follow this  
Followers 0
Wulfmann

Correctly casting serialised objects

12 posts in this topic

Hi guys,

 

I am trying to implement a network architecture in java for a small project etc., however I am unsure of what the design pattern/best-practice is for correctly identifying objects, which are coming through as bytes via DatagramPackets, so that I can then make the appropriate cast where necessary and reconstruct them.

 

The objects that are being communicated are various messages which will make up the client/server communication protocol.

 

I managed to dig up an old thread discussing this same issue and the agreed solution seems to be a dispatcher. Is this the best way of going about this?

 

I decided to give it a try as it seems like a decent solution. My message classes implement the Externalizable interface and I have roughly followed the design discussed in said old thread below:

 

http://www.gamedev.net/topic/634511-concept-of-a-basic-custom-packet-in-use-for-sending-data-to-server-and-client-back-and-forth/

 

In particular the last 2 posts (really 1 post but with the second-last post quoted):

 

 

 

 

I was worried that I have to write a few packets entirely in bytes. What a relief.


I don't think that's a bad thing. You can model the contents of packets as structs/classes, and then actually generate the reading/writing to/from byte arrays as functions, perhaps on a general interface that each of your messages implement.

reading/writing fields of a message isn't that hard to do manually, and gives you great control over how your protocol grows.

class WhisperMessage extends Message {
public int toPlayer;
public string text;
private static const int code = 10; // should not conflict with any other message class
private static const registration = new MessageRegistration<WhisperMessage>(code); // for finding class on receipt
public WhisperMessage(int p, string t) {
    super(code);
    toPlyaer = p;
    text = t;
}
protected void writeData(ByteStream strm) {
    strm.writeInt(toPlayer);
    strm.writeString(text);
}
protected void readData(ByteStream strm) {
    toPlayer = strm.readInt();
    text = strm.readString();
}
};

Class Message would have public read/write functions that take care of also adding the type code and length fields to the output stream, as well as static functions to read message instances from a byte stream (you need to register the code with some dipatcher).

Also, please apologize if my Java is not quite syntactically correct; it's been several years since I last wrote any.

 


Better yet, make the Message class realize the Externalizable interface, which provide those public read and write methods and is seamless pluggable to Java's default serialization framework.

http://docs.oracle.com/javase/7/docs/api/java/io/Externalizable.html

 

 

I am however confused as to how I would implement the dispatcher (and to a degree the Message superclass - what does the constructor do with the code?) described in the second last post (I consider myself somewhat a beginner, especially with serialisation) and was hoping someone could elaborate?

 

tl;dr:

  • What is the best (or at least an elegant) way of correctly identifying and reconstructing serialised objects?
  • How would one link up the Message implementation, described in the post above, to a dispatcher?

Cheers,

Wulf

0

Share this post


Link to post
Share on other sites
Generally all messages should have a header. If your game is TCP-based it can be as simple as a single integer that identifies the type data, if it is UDP based it will include the type of data plus a sequence number and potentially additional useful information.

You get to define what each of those looks like.

Personally I prefer the message format: { u32 size, u32 ID, u64 sequence number, payload }

Attempting to blindly cast the data directly to an object is perhaps one of the worst ways to handle deserialization. Apart from potentially transmitting unnecessary data, many languages and designs also have concerns about proper construction, proper alignment, data location, memory fragmentation, and so on. Also very important, you need to validate your data; you really need to detect things like an attacker sending an extremely long (or negative length) string.

So assuming the format above in a TCP message stream, you would wait until 16 bytes are available in the stream, and then extract them. The first for bytes are the size of the payload, the next four bytes are the way to interpret the payload, the next 8 bytes are the sequence number of the packet which is useful for debugging. Then you wait until the size of the payload is available in the stream. When at least that much content has arrived, you extract that many bytes and send the raw data as either a byte stream or byte buffer to a function that corresponds to the message number.

So let's say you got a message that starts out { 21, 10, 2471, ... } This tells you to wait until 21 bytes are ready in the stream. When they arrive they are packet type 10, which in your case might be 'player changed their name' message. So you dispatch it to a function. This could be through a big switch statement or through components registering their accepted message types with the system. Either way it ends up calling a function, perhaps OnPlayerChangedNameMessage( messageSize, payload ). Then inside that function you extract how long the new name is, validate the value (eg that player actually exists, positive string length, length matches the size of the payload, length within the size allowed by the game, etc.) and then extract the new player's name and validate it (only valid visible characters) and finally pass it along to the proper game component.

Does that answer the question well enough?
2

Share this post


Link to post
Share on other sites

Hi frob,
 
My game is UDP-based, sorry I didn't explicitly mention that. I understand the theory behind network communication and agree with your idea of such a message format.
 
However, to my knowledge java abstracts that kind of lower level detail by providing network classes for both TCP and UDP-based communication. In the case of UDP, there is the DatagramPacket class which contains a byte array (of the payload) and length.
 
To reconstruct the byte array into anything meaningful (and following the Externalizable convention) I need to instantiate an object which will read the bytes before I can do anything else. See the following code:

	public void receive()
	{
		Message msg;
		byte[] buffer = new byte[1024];
		try {
			DatagramPacket packet = new DatagramPacket(buffer, buffer.length);
			socket.receive(packet);
			msg = deserializeMessage(packet.getData());

			netMan.processMessage(msg);
		} catch (Exception e) {
			e.printStackTrace();
		}
	}

	private Message deserializeMessage(byte[] ba) throws Exception {
		ByteArrayInputStream bais = null;
		ObjectInputStream ois = null;
		Message temp = null;
		Message msg = null;
		try {
			bais = new ByteArrayInputStream(ba);
			ois = new ObjectInputStream(bais);
			msg = new Message();
			msg.readExternal(ois);
		} catch (Exception e) {
			e.printStackTrace();
		} finally {
			try {
				if (ois != null)
					ois.close();
			} catch (Exception e){
				e.printStackTrace();
			}
		}
		return msg;
	}

In this case I am instantiating a Message object, which is the highest level parent of all my different messages.

 

I have just thought of a solution where I continue (cannot start again as I have already read some of the bytes) reconstructing the appropriate message based on the code already read in the Message object. However, I am unsure if this is a good option as the parent Message class would be coupled with the message identification process...

0

Share this post


Link to post
Share on other sites

I'm not a regular Java user, but have a quick read of the comments on ObjectInputStream here, and specifically the first example:

 

http://docs.oracle.com/javase/7/docs/api/java/io/ObjectInputStream.html

 

Note that they are not creating the objects or calling readExternal on them themselves. The are instead calling readObject on the ObjectInputStream which automagically returns the correct type. This assumes that you wrote the objects in on the other end using ObjectOutputStream, so that its metadata will be as expected. I expect you will end up with something simple like:

Message msg = ois.readObject();
Edited by Sprangle
0

Share this post


Link to post
Share on other sites

Note that ObjectOutputStream/ObjectInputStream is essentially an implementation of what frob has described for you above, it's just they've done some of the work of reading and writing message types for you. Depending on your needs this might be OK, or you might want to get your hands dirtier and take full control of the data.

 

As for a dispatcher, that could mean lots of different things. This is a simplified version of how mine works in C#:

  • All messages are derived from Message.
  • The network service deserializes a message (as above) and verifies that it is valid.
  • It then passes the message to the dispatcher, which in my case is called MessagingService.
  • Various other parts of the system have registered themselves with the messaging service, specifying a message type (e.g. ChatMessage) and a callback (e.g. HandleChatMessage).
  • MessagingService checks the type of the message, looks up the list of registered callbacks for that type, and calls them each in turn.

From there you can add whichever improvements you want. Some of mine include meta data on which client sent the messages, security information, and a bit of reflection to wire up my handlers automatically for me. My network is also registered with the MessageService so that the server can use it to post messages back to clients, without ever having to talk to the networking layer.

 

What this all gives you is a network and dispatching layer that is nicely decoupled from any details of your messages or what they are used for. It also lets your game or application forget about the networking side and just deals with neat strongly typed messages. My solution is more focused on convenience for the programmer that performance, but you get the idea.

Edited by Sprangle
1

Share this post


Link to post
Share on other sites
Too much. You need a factory that can select the appropriate class based on the int type code. This class can instantiate the specific message class using a parameterless constructor. A switch statement is not a terrible way to go when selecting the class to create. Then the payload is given to the message class's deserialize method. That's all.
1

Share this post


Link to post
Share on other sites

So one problem with using Serialization for a game is that it is difficult to force implementations on your users.  Serial objects can be get very picky about what version of Java is running, and what version of class you are casting to/from.  Imagine a situation where you've got to fix a bug, but that change breaks everyone's code unless they upgrade their Java or upgrade their game.  I don't know about you, but I like to upgrade my stuff when I'm ready.

 

If you do the data yourself, you have complete control of updates, and the server can insert default data for new features, handle many versions of clients, many versions of code, and lots of other issues that will come up. 

1

Share this post


Link to post
Share on other sites

As always there are trade offs you have to consider for your own situation. I would consider a call to readObject to be much simpler and easier to maintain than a big switch statement. On the downside you a lot of control, and Glass_Kinfe notes some Java versioning issues I was not aware of.

 

For the record I work in C# and don't use the built in serialization, I have the equivalent of the big switch statement.

0

Share this post


Link to post
Share on other sites

I want to note that readObject does not actually return the right type. Java has no support for dynamic return types, and readObject will just return an Object which you have to cast to the proper class through.. a huge ass switch statement (or equivalent construct e.g. factory pattern or an instance map).

 

For prototyping Java serialization is pretty awesome, but obviously for a robust project you'd want to provide your own serialization methods independent of the JVM runtime, as Glass Knife mentioned.

1

Share this post


Link to post
Share on other sites

Correct, that would be the "dispatcher" originally asked about.

 

...a huge ass switch statement (or equivalent construct e.g. factory pattern or an instance map)...

0

Share this post


Link to post
Share on other sites

Thanks for all the replies and a great discussion! I ended up implementing a Factory class as smr and Bacterius have suggested and believe it is a suitable solution for now. I am trying to hack up the network layer reasonably quickly so I can begin to implement the more exciting details of the game, but at the same time I want it to be a reasonable solution that follows some common conventions.

 

This is why I have opted to use the Externalizable interface rather than retain full control over the data.

 

Also, thank you for your explanation of the Dispatcher Sprangle, I will most likely end up implementing something of this sort when I move onto message handling.

0

Share this post


Link to post
Share on other sites

You should never go through pain Object IMO. If you manufacture objects from a factory, make them derive from IFactoryOutput. Make the interface have whatever polymorphic functionality you need, such as "handleIncomingMessage" or whatever.

switch() is a signal that you're doing something wrong.

Edited by hplus0603
0

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0