Sign in to follow this  

[java] serialized data for network transfer, and reflection

This topic is 3627 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

When we send data between client and server, I imagine that the typical way is to send a chunk of raw binary data, plus something describing what type of object we just passed. Then the server can use the id to use the correct code to understand the encoded data, typically by populating it into a class. Am I along the right lines here? So specifically with java, it seems reflection could be nice here? i.e the client sends an id of 123 plus some data, the server looks up 123 and finds it's mapped to "GunFire" and then creates a GunFire object using reflection, passing the data as construction argument. This would avoid nasty switch statements and could be quite elegant from an architectural point of view, but I don't have much knowledge of how reflection performs. I have been told before you should only use it sparingly...

Share this post


Link to post
Share on other sites
As for Java, you can actually leave out the initial ID. Just send the actual name of the class and then use:
  java.lang.Class example = java.lang.Class.forName( "ClassName" ); 

There are similar methods to obtain fields and methods but I don't recall them right now.^^

Another thing you might want to have a look at is RMI (remote method invocation). It lets you treat objects across the network as if they were local. Keep in mind, though, that it's pretty time consuming.

Share this post


Link to post
Share on other sites
The client and the server are written in totally different languages. So RMI is way to much of a pain, and probably not good performance-wise anyway.

As far as reflection goes, is it really slow? How much overhead is there on top of doing a big if-else block on the class name and creating objects normally?

Share this post


Link to post
Share on other sites
Quote:
So specifically with java, it seems reflection could be nice here?


Oh looky, a shiny new hammer.

Quote:
This would avoid nasty switch statements


Yea, switch statements are the wrong choice here in the first place, reflection or not.

If you mean reflection like this:
HashMap<int, Constructor> messageMap
then yes, it's suitable.

But if you're implying that reflection be used for anything more, such as searching the code-base to find a particular class matching a certain signature representing an event, then it's a big no-no. Your message interfaces and class definitions will not be changing during run-time - you won't be providing and compiling new implementation code. Even if you are - you receive tens of thousands of network events per second, and you might need to issue such update once every few hours, days, weeks or months. Clearly, the wrong approach.

Java is incredibly hostile to RAII, making it essentially impossible. This forces you into polymorphism to achieve the above. A typical network message will likely use command pattern, or single class for everything.

interface NetworkMessage
{
virtual void execute( Game g );
}

class GunFireMessage implements NetworkMessage
{
GunFireMessage ( Stream s );
virtual void execute( Game g )
{
g.getPlayer(this.id).fireGun();
}
}

...

Constructor c = messageMap.get(stream.readId());
NetworkMessage msg = (NetworkMessage)c.invoke(stream);
msg.execute(game);


This would be much easier if Java's generics were completely broken in design, so this is as good as it gets.

Share this post


Link to post
Share on other sites
I mean reflection like

void networkMessageRecieved(String message)
{
Object o = Class.forName( message ).newInstance();
requestHandler.handle(o);
}

I'm not sure if this is one of the things you describe as suitable or bad? Obviously there would be some message data involved but I left that out for simplicity.

Share this post


Link to post
Share on other sites
Quote:
Original post by d000hg
I mean reflection like

*** Source Snippet Removed ***I'm not sure if this is one of the things you describe as suitable or bad? Obviously there would be some message data involved but I left that out for simplicity.


Congratulations - you are now certified enterprise-grade developer :)

Please don't do that. It's redundant and incredibly unsafe.

What happens if client sends "new SqlStatement("DROP DATABASE;");"?

Why are you doing Class.forName on Message? If you have the .class already, you can assign it an ID during application startup. If you don't have it - what then? Send it via stream as bytecode? Do you really want to turn your server into a botnet?

And if you do have all .class definitions already, what are you repeating an insanely expensive operation on every message? Even if you predetermine Class from a string and cache that, the overhead is 2x the cost of virtual function call. Even more - you receive an Object, which means you'll need to use a sequence of if statements (if (o instanceof GunFireMessage) ...).

And even assuming you do get bytecode class definitions over network - how will you handle them. What will you do with "MyFooBarMessage"? Send handler alongside? The horror....

Which brings us back to original problem.

You have a set of messages with 1-to-1 mapping to network IDs. Solve this problem, and you're done with it.

Reflection in Java is one of the features that has been abused in IT world so horribly, that it created perversions beyond belief. Reflection, just like singletons, has its uses. But it's not until you encounter a 30 million line application where no classes are compile-time dependant on each other, but rather connected through configuration files with classes streamed over the network that you realize how wrong that is.

If you feel the need for dynamic message registration, then use a map into which handlers install themselves. Each message type generates its own ID (crc, perhaps even SerialID) which is then used as key in a map that looks up the constructor.

Reflected creation however is very much out of place in a *network* handler, where you definitely do not want flexibility. You want rigidly controlled handler that discards anything that even by a single bit deviates from expected input.

Share this post


Link to post
Share on other sites
Thanks for your input... like I said at the start I had doubts myself that reflection was the right tool. It seems like my hunch was well-founded. I guess the traditional approach is either a big switch or if/else block (not pretty but very common) or to register a very simple factory against each accepted message type?

Share this post


Link to post
Share on other sites
The problem with reflection is that you lose compile-time check. With ~200 messages (conservative estimate) you are almost guaranteed that someone will mis-spell at least one class name.

And you will never know of this problem until the message is triggered - or never is.

--vvv-- don't do the following --vvv---

The solution to this is to write unit tests. They make sure that every message is properly configured. This is tested by hard-coding valid messages into unit test, and then using MyMessage.class.getName() to construct a message, then passing it to handler. If errors occur, unit tests fail.

Obviously, now the message names are hard-coded. Which is bad. So messages are instead stored in a database. Hibernate is used to map those during run-time, stream them into unit tests and start the server.

But database connection may fail. So mock objects are used for both, message handler and database to ensure that it works.

New problem arises, since developers now need to manually update external database whenever they add a new message. So Maven is brought it. On every build, it scans through the entire codebase, searches for Messages, and generates both, mock messages as well as real ones. It also inserts them into database.

Unit tests are provided for this Maven tasks to make sure it works properly.

------

Can you guess what declaring a message in a .java file does? It solves all of those problems and warns you during compile-time if anything mismatches (about 0.5ms response time, or in real-time if using IDE with code completion).

Yet the brain dead unit test approach is used by just about every big-league consultancy to solve problems that don't even exist using millions of lines of code.

The above example is sadly not made up. There are literally thousands of frameworks, some by very established names and brands that do exactly that. Debugging and testing such applications is impossible by definition, and relies on code not getting executed for everything to work.

If you need to use Class.forName, use the MyObject.class.newInstance() to provide compile-time check.

To register a message with dispatcher, just pass x.class to it. Handler then calls getName(), generates an ID string from that, installs constructor and creates the lookup map.

Very simple, very easy, no globals, no coupling. Even more, message handler doesn't need to include any message .java files, nor know about them. Almost identical to your solution, but type-safe, compile-time resolved and very easy to maintain, since your SpecificHandler registers itself (knows about message type, as well as logic).

Share this post


Link to post
Share on other sites

This topic is 3627 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this