Converting Objects to bytes? C#

Hey,

You can use a BinaryFormatter to serialize into a MemoryStream, from which you can get the bytes.

Good luck,
Sijmen Mulder

Hrm,

I saw that option when I was searching around. It sounds rather expensive processor wise.

Would it be more efficient just to take a byte buffer and drop in your values (which would normally be in the object), send it, then parse the values out again on the other side?

If you're sending enough data over the Internet that serialization shows up on a profile, you have a better network connection than anyone else I know :-)

Really, if you're sending 5 kilobytes a second per client, then that's one-millionth of the available memory bandwidth of a typical PC today. Even if you're CPU bound in serialization, rather than memory bound, with an extravagant ratio of 100:1, serialization won't be noticed on a profile until you have 1,000 clients, and even then you have 10x headroom.

well, but when you put your data in your own bytestream you can compress and optimize it a lot better than just to serialize everything.

Ok,

(Sorry, I switched names from GeekPlusPlus. Obviously i'm using C# now, it was habbit to use that username. Purely accidental.)

So I setup a serialization system.... goes something like this...



Both client and server have the exact same Packet.cs file where TextPacket inherits from abstract Packet which has an abstract Handle() function.

The problem is when the bytes get to the server app and it tries to deserialize I get an exception that says:z

"Cannot find the assembly ConsoleTest, Version=, Culture=neutral, PublicKeyToken=null."

Now ConsoleTest is the name of my test client app. But I've tried making the namespace the same, the function that calls it the same, and even the project name the same as the server, and it still always says the same exception.

Is serialization only going to work for p2p where the app is actually the same?

Hope someone can help here.

Well nevermind,

I figured it out. And for those of you who are curious...

The assembly of your classes is part of the deffinition of them. Therefore even if your class has the same name and exists in the same namespace, if they are in two different assemblies then they are two different classes.

To solve the problem, instead of using the same cs file Packets.cs in both projects. I just linked the test client to the network dll for the server. this way they are using the exact same packet classes.

Works perfectly.

Now, if someone wants they could tell me if [Serializable()] is enough? or should I actually inherit from ISerializable as well? Does it matter if you're going to serialize all the members anyway?

EDIT: And now that I've done some tests, serialization is sending a 161 byte packet just to send a uint packet number and the letter "H". Lets see... that's 2+8 = 10 bytes, that's only 151 bytes of overhead.... yeouch...



[Edited by - GeekSharp on March 14, 2005 2:33:56 AM]

Posting a roundup of how you solved your problem always deserves a rating up in my book [wink]

Yes, I couldn't make serialization work well either; I wound up adding AddPayload(...) methods to my Packet class instead; worked out pretty well with recycling packets and buffers - http://www.lidgren.net/programming/network/

I've just finished dealing with the exact same problems you're dealing with here. Figuring out serialization, using the same class .DLL for both client and server, and tons of byte overhead.

I fixed it all by throwing serialization out of the network code entirely. Was just too frustrating and not worth the trouble. It's much easier to just construct a basic tokenization/detokenization method for each object you're wanting to send. There are a few ways to do this, and I, being lazy, just opted for the most obvious.

Instead of seralizing and sending this:

**
A Spaceship Object
Object ID = 12
X = 39.3
Y = 204.9
FacingDegree = 175
HeadingDegree = 179
Speed = 7
Thrusting = true
Rotate Left = false
Rotate right = false
**

I just had the object pack itself into a nice little string:
"12|39.3|204.9|175|179|7|100"

The 100 at the end representing true, false, false. Another method in the class takes the tokenized string and unpacks it in the same order to the proper variables. All I have to do is just fire that short string to the client and it takes care of the rest.

So, that was my solution. Some people will probably disagree, but my motto is to just do whatever works. I saved myself who knows how many programming hours trying to optimize net object serialization that would basically be doing the exact same thing.

You can even reduce this more if you know a certain variable will always be the same amount of digits. For example you have 8 class variable you want to send, all of them will never be above or below two digits, so your entire packet data would just look like: "1049328573957629", and you know every two digits should become a new variable.

Cyric

i've got 'another' solution (i try to not use the word better, as it differs)

but ive done some MMORPG emulators in C#, and i've found the best way to convert objects to bytes (and back again!) easily is this little snizzle of code here.

oh.. and your class/struct must be serializable

[Serializable()]
public struct Packet
{
public int PacketId;
public char[] CharacterName = new char[20];
public bool IsAlive;
}

then you pass a 'Packet' instance into this method, and it gives you back a byte array of your Packet class (with zero overhead!):

public static byte[] SerializeExact( object anything )
{
int structsize = Marshal.SizeOf( anything );
IntPtr buffer = Marshal.AllocHGlobal( structsize );
Marshal.StructureToPtr( anything, buffer, false );
byte[] streamdatas = new byte[ structsize ];
Marshal.Copy( buffer, streamdatas, 0, structsize );
Marshal.FreeHGlobal( buffer );
return streamdatas;
}

and the matching Deserialize:

public static object RawDeserialize( byte[] rawdatas, Type anytype )
{
int rawsize = Marshal.SizeOf( anytype );
if( rawsize > rawdatas.Length ) return null;
IntPtr buffer = Marshal.AllocHGlobal( rawsize );
Marshal.Copy( rawdatas, 0, buffer, rawsize );
object retobj = Marshal.PtrToStructure( buffer, anytype );
Marshal.FreeHGlobal( buffer );
return retobj;
}

its very fast, and very useful for packing packets!!

Hope i've been of help!

Wow, two more good suggestions.

What I find really interesting is the idea of tokenized packets... With taht method you could write .ToString() methods on all the objects to conver them to a transferable state... You get a few extra bytes with the pipe characters though but it could be well worth it. Everytime you transfer a string you'd have a byte for the number of byte quads or byte octets anyway. I will probably try this out!

Thanks!

Strings are at least 2x bigger than you need to use.

I would suggest using .toBinary() which generates a known number of bytes (rather than characters). After all, -2000000000 is 11 characters, but a signed int is only 4 bytes.

After you do that, you can go on to do domain-specific compression of the data.

I should of added on the tokenization that I send a byte array using Text encoding to ASCII derived from the string. If the string is "12345", it is only actually sending a 5 byte array, and not a string object. You don't want to be sending an actual string object as that would basically be the same as what you were trying above. Sorry for not going into more depth on this.

I do like hplus0603's suggestion above, and will probably try to work it into my own code.

Hello,

I did quickly realize the flaw with the tokenized strings. large numbers being far more costly then they should be and have gone to a rather simplistic message system and have .ToByte() functiosn written for my server messages which are also used internally to message different systems.

The biggest problem I'm facing now is (and maybe I should make this a new thread...) that it doesn't seem very OOPish for one object to get a packet from the network manager, get the first byte(or two) out of the packet which refer to it's packetID, and then run though a huge if block to decide what messages to create and send to which systems pretaining about the received info from a client.

Right now i'm using a message centre to pass messages between systems (objects). For example, comabt logic might send a MoveEntityMessage to the message centre. that would fire off a listening delegate which EntityManager registered with the message centre and so EntityManager would do whatever he does with the MoveEntityMessage.

Does it make sense, OOP wise, to have a ReceivedPacketMessage which holds the byte data. All systems which could possibly want a packet, would register with the message centre to recieve this message type. When a packet comes in and gets pushed to the message centre, all these sytems would get the data. They would check if the packetID is one of THEIRS, and if not... just ignore it.

I think that's about the best system I can think of for converting packets to useable structures without having a "know-it-all" object.

ideas? comments? flames?

You'll still need an know-it-all enum for the PacketIDs. Switching on packetid might not be completely OOP but it's fairly efficient...

You don't necessarily need a know-it-all enum -- you could configure the packet Ids by name in an XML file, and register packet type handlers by name at runtime, for example.

Each packet handler ("NewEntityHandler," "UpdateEntityHandler," "GameOverHandler," or whatever) would execute in an environment where they know what to do with received data. For example, I'd assume that a map of entities would be available to all of them, so that the UpdateEntityHandler can decode the indicated entity Id (or Ids) and forward the appropriate encapsulated data to the network data interface of each entity indicated.

So, if you want to decompose your network handling a bit, you'd have:

1) Socket receives packet, forwards to Packet Management
2) Packet Management reads necessary data out of packet (sequence number for acks, server timestamp, or whatever), and then enters a loop
3) In the loop, find the PacketHandler for the indicated packet type of the next piece of data of the packet; dispatch the data pointer/size to that PacketHandler.
4) The PacketHandler decodes as much data as necessary from the data pointer, and does what it needs to. It may dispatch in turn to Entities (where specific subsystems may be viewed as entities as well).
5) Dispatched-to Entities/subsystems keep decoding data as necessary and as indicated in the packet.
6) When the dispatch of one embedded message is complete, the amount of data consumed is returned to the packet manager, which repeats the loop after advancing the pointer, until there are no more messages in the packet.

You should probably be fairly paranoid in your packet reading/decoding, paying attention to cases where bad/corrupt data or too-short packets would cause you to read or write off the end of some buffer. Detect that before doing the operation, and return/throw an error instead.

Get .NET Reflector (a spectacular free tool to have anyway) from here: http://www.aisto.com/roeder/dotnet/

Then aim Reflector at your Microsoft.DirectX assembly's DXHelp class. The methods of this class are used to convert objects into byte arrays or directly into unmanaged memory. This should give you an idea of how to do it on your own.