Jump to content

  • Log In with Google      Sign In   
  • Create Account


Correctly casting serialised objects


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
12 replies to this topic

#1 Wulfmann   Members   -  Reputation: 115

Like
0Likes
Like

Posted 13 September 2013 - 10:03 PM

Hi guys,

 

I am trying to implement a network architecture in java for a small project etc., however I am unsure of what the design pattern/best-practice is for correctly identifying objects, which are coming through as bytes via DatagramPackets, so that I can then make the appropriate cast where necessary and reconstruct them.

 

The objects that are being communicated are various messages which will make up the client/server communication protocol.

 

I managed to dig up an old thread discussing this same issue and the agreed solution seems to be a dispatcher. Is this the best way of going about this?

 

I decided to give it a try as it seems like a decent solution. My message classes implement the Externalizable interface and I have roughly followed the design discussed in said old thread below:

 

http://www.gamedev.net/topic/634511-concept-of-a-basic-custom-packet-in-use-for-sending-data-to-server-and-client-back-and-forth/

 

In particular the last 2 posts (really 1 post but with the second-last post quoted):

 

 

 

 

I was worried that I have to write a few packets entirely in bytes. What a relief.


I don't think that's a bad thing. You can model the contents of packets as structs/classes, and then actually generate the reading/writing to/from byte arrays as functions, perhaps on a general interface that each of your messages implement.

reading/writing fields of a message isn't that hard to do manually, and gives you great control over how your protocol grows.

class WhisperMessage extends Message {
public int toPlayer;
public string text;
private static const int code = 10; // should not conflict with any other message class
private static const registration = new MessageRegistration<WhisperMessage>(code); // for finding class on receipt
public WhisperMessage(int p, string t) {
    super(code);
    toPlyaer = p;
    text = t;
}
protected void writeData(ByteStream strm) {
    strm.writeInt(toPlayer);
    strm.writeString(text);
}
protected void readData(ByteStream strm) {
    toPlayer = strm.readInt();
    text = strm.readString();
}
};

Class Message would have public read/write functions that take care of also adding the type code and length fields to the output stream, as well as static functions to read message instances from a byte stream (you need to register the code with some dipatcher).

Also, please apologize if my Java is not quite syntactically correct; it's been several years since I last wrote any.

 


Better yet, make the Message class realize the Externalizable interface, which provide those public read and write methods and is seamless pluggable to Java's default serialization framework.

http://docs.oracle.com/javase/7/docs/api/java/io/Externalizable.html

 

 

I am however confused as to how I would implement the dispatcher (and to a degree the Message superclass - what does the constructor do with the code?) described in the second last post (I consider myself somewhat a beginner, especially with serialisation) and was hoping someone could elaborate?

 

tl;dr:

  • What is the best (or at least an elegant) way of correctly identifying and reconstructing serialised objects?
  • How would one link up the Message implementation, described in the post above, to a dispatcher?

Cheers,

Wulf



Sponsor:

#2 frob   Moderators   -  Reputation: 20359

Like
2Likes
Like

Posted 14 September 2013 - 07:56 PM

Generally all messages should have a header. If your game is TCP-based it can be as simple as a single integer that identifies the type data, if it is UDP based it will include the type of data plus a sequence number and potentially additional useful information.

You get to define what each of those looks like.

Personally I prefer the message format: { u32 size, u32 ID, u64 sequence number, payload }

Attempting to blindly cast the data directly to an object is perhaps one of the worst ways to handle deserialization. Apart from potentially transmitting unnecessary data, many languages and designs also have concerns about proper construction, proper alignment, data location, memory fragmentation, and so on. Also very important, you need to validate your data; you really need to detect things like an attacker sending an extremely long (or negative length) string.

So assuming the format above in a TCP message stream, you would wait until 16 bytes are available in the stream, and then extract them. The first for bytes are the size of the payload, the next four bytes are the way to interpret the payload, the next 8 bytes are the sequence number of the packet which is useful for debugging. Then you wait until the size of the payload is available in the stream. When at least that much content has arrived, you extract that many bytes and send the raw data as either a byte stream or byte buffer to a function that corresponds to the message number.

So let's say you got a message that starts out { 21, 10, 2471, ... } This tells you to wait until 21 bytes are ready in the stream. When they arrive they are packet type 10, which in your case might be 'player changed their name' message. So you dispatch it to a function. This could be through a big switch statement or through components registering their accepted message types with the system. Either way it ends up calling a function, perhaps OnPlayerChangedNameMessage( messageSize, payload ). Then inside that function you extract how long the new name is, validate the value (eg that player actually exists, positive string length, length matches the size of the payload, length within the size allowed by the game, etc.) and then extract the new player's name and validate it (only valid visible characters) and finally pass it along to the proper game component.

Does that answer the question well enough?
Check out my personal indie blog at bryanwagstaff.com.

#3 Wulfmann   Members   -  Reputation: 115

Like
0Likes
Like

Posted 14 September 2013 - 09:25 PM

Hi frob,
 
My game is UDP-based, sorry I didn't explicitly mention that. I understand the theory behind network communication and agree with your idea of such a message format.
 
However, to my knowledge java abstracts that kind of lower level detail by providing network classes for both TCP and UDP-based communication. In the case of UDP, there is the DatagramPacket class which contains a byte array (of the payload) and length.
 
To reconstruct the byte array into anything meaningful (and following the Externalizable convention) I need to instantiate an object which will read the bytes before I can do anything else. See the following code:

	public void receive()
	{
		Message msg;
		byte[] buffer = new byte[1024];
		try {
			DatagramPacket packet = new DatagramPacket(buffer, buffer.length);
			socket.receive(packet);
			msg = deserializeMessage(packet.getData());

			netMan.processMessage(msg);
		} catch (Exception e) {
			e.printStackTrace();
		}
	}

	private Message deserializeMessage(byte[] ba) throws Exception {
		ByteArrayInputStream bais = null;
		ObjectInputStream ois = null;
		Message temp = null;
		Message msg = null;
		try {
			bais = new ByteArrayInputStream(ba);
			ois = new ObjectInputStream(bais);
			msg = new Message();
			msg.readExternal(ois);
		} catch (Exception e) {
			e.printStackTrace();
		} finally {
			try {
				if (ois != null)
					ois.close();
			} catch (Exception e){
				e.printStackTrace();
			}
		}
		return msg;
	}

In this case I am instantiating a Message object, which is the highest level parent of all my different messages.

 

I have just thought of a solution where I continue (cannot start again as I have already read some of the bytes) reconstructing the appropriate message based on the code already read in the Message object. However, I am unsure if this is a good option as the parent Message class would be coupled with the message identification process...



#4 Sprangle   Members   -  Reputation: 128

Like
0Likes
Like

Posted 16 September 2013 - 05:40 AM

I'm not a regular Java user, but have a quick read of the comments on ObjectInputStream here, and specifically the first example:

 

http://docs.oracle.com/javase/7/docs/api/java/io/ObjectInputStream.html

 

Note that they are not creating the objects or calling readExternal on them themselves. The are instead calling readObject on the ObjectInputStream which automagically returns the correct type. This assumes that you wrote the objects in on the other end using ObjectOutputStream, so that its metadata will be as expected. I expect you will end up with something simple like:

Message msg = ois.readObject();

Edited by Sprangle, 16 September 2013 - 05:41 AM.


#5 Sprangle   Members   -  Reputation: 128

Like
1Likes
Like

Posted 16 September 2013 - 05:58 AM

Note that ObjectOutputStream/ObjectInputStream is essentially an implementation of what frob has described for you above, it's just they've done some of the work of reading and writing message types for you. Depending on your needs this might be OK, or you might want to get your hands dirtier and take full control of the data.

 

As for a dispatcher, that could mean lots of different things. This is a simplified version of how mine works in C#:

  • All messages are derived from Message.
  • The network service deserializes a message (as above) and verifies that it is valid.
  • It then passes the message to the dispatcher, which in my case is called MessagingService.
  • Various other parts of the system have registered themselves with the messaging service, specifying a message type (e.g. ChatMessage) and a callback (e.g. HandleChatMessage).
  • MessagingService checks the type of the message, looks up the list of registered callbacks for that type, and calls them each in turn.

From there you can add whichever improvements you want. Some of mine include meta data on which client sent the messages, security information, and a bit of reflection to wire up my handlers automatically for me. My network is also registered with the MessageService so that the server can use it to post messages back to clients, without ever having to talk to the networking layer.

 

What this all gives you is a network and dispatching layer that is nicely decoupled from any details of your messages or what they are used for. It also lets your game or application forget about the networking side and just deals with neat strongly typed messages. My solution is more focused on convenience for the programmer that performance, but you get the idea.


Edited by Sprangle, 16 September 2013 - 06:05 AM.


#6 smr   Members   -  Reputation: 1624

Like
1Likes
Like

Posted 16 September 2013 - 07:31 AM

Too much. You need a factory that can select the appropriate class based on the int type code. This class can instantiate the specific message class using a parameterless constructor. A switch statement is not a terrible way to go when selecting the class to create. Then the payload is given to the message class's deserialize method. That's all.

#7 Glass_Knife   Moderators   -  Reputation: 4117

Like
1Likes
Like

Posted 16 September 2013 - 09:43 AM

So one problem with using Serialization for a game is that it is difficult to force implementations on your users.  Serial objects can be get very picky about what version of Java is running, and what version of class you are casting to/from.  Imagine a situation where you've got to fix a bug, but that change breaks everyone's code unless they upgrade their Java or upgrade their game.  I don't know about you, but I like to upgrade my stuff when I'm ready.

 

If you do the data yourself, you have complete control of updates, and the server can insert default data for new features, handle many versions of clients, many versions of code, and lots of other issues that will come up. 


I think, therefore I am. I think? - "George Carlin"
Indie Game Programming

#8 Sprangle   Members   -  Reputation: 128

Like
0Likes
Like

Posted 16 September 2013 - 02:24 PM

As always there are trade offs you have to consider for your own situation. I would consider a call to readObject to be much simpler and easier to maintain than a big switch statement. On the downside you a lot of control, and Glass_Kinfe notes some Java versioning issues I was not aware of.

 

For the record I work in C# and don't use the built in serialization, I have the equivalent of the big switch statement.



#9 Bacterius   Crossbones+   -  Reputation: 8555

Like
1Likes
Like

Posted 17 September 2013 - 03:31 AM

I want to note that readObject does not actually return the right type. Java has no support for dynamic return types, and readObject will just return an Object which you have to cast to the proper class through.. a huge ass switch statement (or equivalent construct e.g. factory pattern or an instance map).

 

For prototyping Java serialization is pretty awesome, but obviously for a robust project you'd want to provide your own serialization methods independent of the JVM runtime, as Glass Knife mentioned.


The slowsort algorithm is a perfect illustration of the multiply and surrender paradigm, which is perhaps the single most important paradigm in the development of reluctant algorithms. The basic multiply and surrender strategy consists in replacing the problem at hand by two or more subproblems, each slightly simpler than the original, and continue multiplying subproblems and subsubproblems recursively in this fashion as long as possible. At some point the subproblems will all become so simple that their solution can no longer be postponed, and we will have to surrender. Experience shows that, in most cases, by the time this point is reached the total work will be substantially higher than what could have been wasted by a more direct approach.

 

- Pessimal Algorithms and Simplexity Analysis


#10 Sprangle   Members   -  Reputation: 128

Like
0Likes
Like

Posted 17 September 2013 - 03:46 AM

Correct, that would be the "dispatcher" originally asked about.

 

...a huge ass switch statement (or equivalent construct e.g. factory pattern or an instance map)...



#11 TheChubu   Crossbones+   -  Reputation: 4118

Like
0Likes
Like

Posted 17 September 2013 - 03:51 AM

You could do, say:

obj.getClass().cast(obj);

 

Dunno what would accomplish but it can be done (and I find it rather funny :D )

 

In any case, here is an IBM article about Java serialization, might help http://www.ibm.com/developerworks/library/j-5things1/


"I AM ZE EMPRAH OPENGL 3.3 THE CORE, I DEMAND FROM THEE ZE SHADERZ AND MATRIXEZ"

 

My journals: dustArtemis ECS framework and Making a Terrain Generator


#12 Wulfmann   Members   -  Reputation: 115

Like
0Likes
Like

Posted 17 September 2013 - 05:28 AM

Thanks for all the replies and a great discussion! I ended up implementing a Factory class as smr and Bacterius have suggested and believe it is a suitable solution for now. I am trying to hack up the network layer reasonably quickly so I can begin to implement the more exciting details of the game, but at the same time I want it to be a reasonable solution that follows some common conventions.

 

This is why I have opted to use the Externalizable interface rather than retain full control over the data.

 

Also, thank you for your explanation of the Dispatcher Sprangle, I will most likely end up implementing something of this sort when I move onto message handling.



#13 hplus0603   Moderators   -  Reputation: 5174

Like
0Likes
Like

Posted 17 September 2013 - 01:33 PM

You should never go through pain Object IMO. If you manufacture objects from a factory, make them derive from IFactoryOutput. Make the interface have whatever polymorphic functionality you need, such as "handleIncomingMessage" or whatever.

switch() is a signal that you're doing something wrong.


Edited by hplus0603, 17 September 2013 - 01:33 PM.

enum Bool { True, False, FileNotFound };




Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS