Archived

This topic is now archived and is closed to further replies.

fractoid

Java serialization and compression overheads vs. UDP

Recommended Posts

Okies, here''s the next step in my quest to know everything pertinent about netcode programming before actually embarking on my netcode. This time it''s somewhat java-dependant, although it could relate to other languages too. I''m planning on sending UDP game updates from the server between 10 and 20 times a second. Each game update is going to consist of an array of 2(n+1) 16-bit integers, where n is the number of players that the client in question (x, y for each player, plus timestamp plus ''game id'' so stray packets don''t make things go weird). I expect no more than 20 players onscreen at once, so say 84 bytes per datagram max, raw. The simplest and most obvious way to do things is just stuff those ints into a byte array and send it, which I was going to do. However, every time I start thinking about doing bit shuffling in Java I realize that I''m thinking like a C++ programmer again (which, let''s face it, is mostly what I am). Sitting back and putting on my unigoggles (like beer goggles, but make theoretically fine solutions look good even if they''re ugly hags in the real world), I realize that what I should be doing is using an ObjectOutputStream, piped through a ZipCompressionStream or whatever it''s called, into a byte buffer output stream thingie, then getting the bytes from that and sending it. Advantages of this are that the data is munged and hence difficult to snoop, and that it means if I want to send a string and some ints instead it''s easy to change. Also, the built-in compression may even result in a smaller datagram than I would previously have made. Disadvantages are that, knowing java, I could easily end up sending far more data than I need to, due to overheads caused by generic-icity. Also, object serialization tends to lock the protocol to java, so I couldn''t, for instance, rewrite the server (or client) in C++. I''m not too worried about that last problem, but I *am* worried about taking up twice as much room as necessary, since I''m trying to squeeze the game''s bandwidth requirements to fit down a 56k modem. I could run some benchmarks and so on to figure it out for myself, but I wondered, with so many knowledgeable people floating around, if it''d been done before and someone already konws?

Share this post


Link to post
Share on other sites
Since your compression sceheme is the well known zip algorihtim, it''s unlikely it would stop anyone from uncompressing it. Also the zip algorithim does not work well for small data blocks. It actually increases their size due to the overhead of the headers, or if it uses an adaptive compression scheme, the data set is too small to get any compression on.

So that would negate the 2 reason for using a compressed stream in this manner. However there might be a good reason to use an object stream, it would simplfy the network protocols, as you are freed from writting packet read/write code which can get tedieous to maintain.

At 20x sec at 84 bytes = 1680 bytes, which is within 56k modem bandwitdth range. Stay within that range and you''ll be ok.

Good Luck!

-ddn

Share this post


Link to post
Share on other sites
ZipCompressionStream is a losing proposition.

Either you re-set the dictionary for each packet, in which case you get very little, if any, real compression.

Or you retain state between packets, but then the first packet to get lost, re-ordered or duplicated will throw off the decoder and the rest of the session is lost.

Share this post


Link to post
Share on other sites
OK, things are looking a bit clearer now... my refined proposition is to use an uncompressed ByteArrayOutputStream(ObjectOutputStream()). Compression was just a random idea - I know that it doesn''t work well on very small data items, but I thought it might be worthwhile given that the java serialization tends to add in some redundant stuff.

The key to making serialization more efficient is to implement Externalizable rather than Serializable, and handle the byte packing myself. ChurchillObjects.com has a good tutorial on serialization, with this and other helpful info. Given that I''ll just be sending a short[50] or so, though, I doubt it''ll be necessary, as I could just send the raw array object.

Hopefully the overhead is small enough, as I''d really like to keep the generality that the ObjectStream provides. I''ll do some tests as soon as I get home, to see exactly what that overhead is.

Thanks for the help.

Share this post


Link to post
Share on other sites
You really don''t want to use the standard Java serializable interface. The only way to know how large the byte array that it serialises to is to serialise it and check the length (this sucks).

Serialising has to add enough data to the stream so that the VM on the far side can completely reconstruct the sent object.

Go with Externalizable and make your own marshalling/unmarshalling functions.

Share this post


Link to post
Share on other sites
quote:
Original post by fractoid
[...]The key to making serialization more efficient is to implement Externalizable rather than Serializable, and handle the byte packing myself. ChurchillObjects.com has a good tutorial on serialization, with this and other helpful info. Given that I''ll just be sending a short[50] or so, though, I doubt it''ll be necessary, as I could just send the raw array object.
[...]

Original post by Silent Bob
[...]Go with Externalizable and make your own marshalling/unmarshalling functions.


OK... I like your idea.

Apparently the overhead for serializing a primitive (non-reference) array is minimal, though... maybe 20 bytes. I found a website where people had done benchmarks, but I can''t find it again. *sigh*.

Share this post


Link to post
Share on other sites
If you are using version 1.4+, stay away from streams altogether. Look into using the nio package (Channels and direct byte buffers). It is much more efficient and does not require the use of a thread per connection.

Share this post


Link to post
Share on other sites
Hmm... this brings us to another of my main points of consternation, which is what runtime to target. I''ve been intending to require only Java 1.1, as many people are limited to the standard IE java plugin still. However, since MS stopped providing that plugin, it''s likely that people who have java support will have at least java 1.4 (or so I assume).

Any idea how I can get stats for which runtime is most prevalent, and the current trends? I''d still like to stick to 1.1 if at all possible, though, because for one example a few of my friends have imacs, which afaik are stuck with the aforementioned sucky 1.1 MS VM.

I''ll give this nio thing a lookee anyway, sounds useful - thanks!

Share this post


Link to post
Share on other sites
quote:
Original post by ddn3
At 20x sec at 84 bytes = 1680 bytes, which is within 56k modem bandwitdth range. Stay within that range and you''ll be ok.



Don''t forget that your data is likely to sit on top of other protocols. In you case, it seems that your data will be sent over UDP 20 times per seconds.

With an overhead of 28 bytes (20 for IP and 8 for UDP), the pure low level protocol overhead would be 560 additional bytes.

This makes a grand total of 2240B/s, which is still reasonable to be carried over a 56K modem.

Share this post


Link to post
Share on other sites
What? he isn''t using the instant tranmission zero overhead protocols? Ofcoruse you are correct kapouille.

-ddn

Share this post


Link to post
Share on other sites
quote:
Original post by fractoid
Hmm... this brings us to another of my main points of consternation, which is what runtime to target. I''ve been intending to require only Java 1.1, as many people are limited to the standard IE java plugin still.


Ah, this is something I wish people would stop doing There''s a little tool that comes with the JDK which can generate code around your applet tags to trigger the download of the latest Java plugin for most browsers. Plop that on your webpage and you''re good to hook for most platforms/browsers.

Share this post


Link to post
Share on other sites