I've just been thinking about this today... What is the fastest way to convert structures to an array of bytes in C#?

Here are two ways I've come up with using generics:

[source lang="csharp"]
public unsafe static byte[] GetBytes<T>(this T value)
where T : struct
{
int size = Marshal.SizeOf(typeof(T));
var buffer = new byte[size];
var pSrc = (byte*)Marshal.AllocHGlobal(size);

Marshal.StructureToPtr(value, (IntPtr)pSrc, true);

fixed (byte* bp = buffer)
for (int i = 0; i < size; ++i)
*(bp + i) = *(pSrc++);

return buffer;
}

public static byte[] GetBytes2<T>(this T value)
where T : struct
{
int size = Marshal.SizeOf(typeof(T));
var buffer = new byte[size];

GCHandle pin = GCHandle.Alloc(value, GCHandleType.Pinned);

try
{
Marshal.Copy(pin.AddrOfPinnedObject(), buffer, 0, buffer.Length);
return buffer;
}

finally { pin.Free(); }
}
[/source]

I really HATE how C# won't allow you to take the address of type T where T: struct... It will let you take the address of a struct, but not a struct used in the context of generics... I think that's stupid, and I wish they'd fix it! AFAIK it's faster to manipulate raw bytes by using pointers and turning off bounds checking than iterating through a managed array the usual way (by indexing). Can anyone confirm/deny this?

So does anyone know the fastest way to do this, and what pros/cons there are to each method? I just ran a test of both of these methods using System.Diagnostics.Stopwatch, and it appears that both are executing in only 1-3 ticks...!? If that is the case it apparently makes little difference, but GetBytes2() is coming in, on average, at 2 ticks whereas GetBytes takes an average of 3 ticks. So I suppose GetBytes2 appears to be about 33.33% faster. I suspect it might make a more considerable difference when much larger data is being worked with or lots of small structures must be serialized in rapid succession (e.g., each frame of a video game).

Anyone have yet an even faster way of doing things? And what about memory efficiency? Which of these ways is more efficient and less wasteful?

That path doesn't end well in the long term.

Any changes to the structure over time --- and there certainly will be evolution of the structure over time --- is not accounted for in your system. Version information is a must.

C# has a built-in serialization system. It includes version information. It allows ways to specify what should be included and what should be excluded. It is automatic. You can write to various formats including binary formats, xml formats, and whatever else you want. It is both fast and efficient.

Do you have any specific reason you aren't using the runtime's serialization functionality?

All I'm trying to do is find a very fast and efficient way to "strip" a structure down to raw bytes. For example, to rapidly create a static vertex buffer or index buffer in the shortest amount of time possible... situations where I don't care about reflecting changes to the data in other places, as it's for one-time use.

I may be incorrect but it's my understanding that this is faster for my limited intents and purposes...it's a very small amount of code that runs in a few ticks, according to test results.

EDIT:

I haven't performed the tests to actually confirm this, but I do believe it would also be faster if I took the address of a structure and cast to byte*, then obtain a byte* to the destination buffer and copy the memory that way. For example:

[source lang="csharp"]
public unsafe byte[] GetBytes(Vector3 vec)
{
var buffer = new byte[Vector3.SizeInBytes];
byte* pSrc = (byte *)&vec;

fixed (byte* baseAddr = buffer)
for (int i = 0; i < Vector3.SizeInBytes; ++i)
baseAddr = pSrc;

return buffer;
}
[/source]

Of course I could be totally wrong about all of this... but that's why I'm hear asking. To learn. :-)

But your proposed setup only works for data types where all of the data is actually in the members, and not in memory pointed to by the members. So you wouldn't be able to serialize arrays for instance, which makes it pretty much useless for serializing vertex buffers. Unless you like declaring vertex buffers like this:


struct VertexBuffer
{
Float3 v0, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10 ... v10236;
}


So you wouldn't be able to serialize arrays for instance, which makes it pretty much useless for serializing vertex buffers. Unless you like declaring vertex buffers like this:

[source lang="csharp"]
public static byte[] GetBytes<T>(this T[] array) where T : struct
{
int size = Marshal.SizeOf(typeof(T));
var buffer = new byte[size * array.Length];

for (int i = 0; i < array.Length; ++i)
{
var bytes = array.GetBytes();
Array.Copy(bytes, 0, buffer, (i * size), bytes.Length);
}

return buffer;
}[/source]

Already been taken care of, and it all seems to perform blazing-fast. :-)

EDIT:

Also, the first GetBytes<T>(...) implementation I showed is, I believe, over-complicated and wastes memory by duplicating things. I think this would actually work better with no need to allocate a new block unmanaged memory, as long as you fix the buffer:

[source lang="csharp"]
static unsafe byte[] _getBytes<T>(T value) where T : struct {

var buffer = new byte[Marshal.SizeOf(typeof(T))];

fixed (byte* pBuffer = buffer)
Marshal.StructureToPtr(value, (IntPtr)pBuffer, true);

return buffer;
}[/source]

EDIT2:

Yes, it does indeed work. I'm sure it's faster than that first implementation I showed. Perhaps it can even be faster than GetBytes2... we'll see...

EDIT:

Ok, I setup the testing code to run 2,000,000 tests of each method and compute the average time each one takes. GetBytes2 wins, but only by less than 0.07 ticks. The changes to GetBytes1 do give it a tiny boost of speed.

I should also mention that before actual testing commences I run one interation of the test and ignore the results. This is to "pre-JIT" the code because the first time it runs it must be JIT'ed and incurs an overhead subsequent calls will not.

Well, I guess I'm already doing things as fast as they can be done. I don't see how anything can be faster than 1-2 ticks, as that's virtually the smallest slice of processor time possible. This code is currently a working part of my engine used to generate raw bytes from structures which is then bound to the GPU as vertex and index buffers, for example. I would've used .NET's Serialization had I wanted to do anything else, like sending data over a network, but in this case I don't need any type information, persistence or anything else... I just need raw, contiguous data for the GPU, and I need it as fast as humanly (well... "machinely") possible. So I guess it's not getting any better or faster than it already is.

Thanks for all the help and your time!