Returning Buffer from C to C#

Started by
7 comments, last by frob 6 years, 8 months ago

I don’t C# very often.

I have a C function that takes a byte array as input and should return a string it generates internally.

Just for starters I generate the string and discard it since I don’t know how to return it to C#.  In C I have this:


int __stdcall ConvertHodgmanKylotanjbadamsfrobRavyneApochPiQSeanMiddleditchrip_offspaceratLightness1024_to_String(const char * thoseGuys);

In C# I have this:
 


[DllImport("Important.dll", CharSet = CharSet.Ansi, CallingConvention = CallingConvention.StdCall)]
private static extern unsafe Int32 ConvertHodgmanKylotanjbadamsfrobRavyneApochPiQSeanMiddleditchrip_offspaceratLightness1024_to_String(byte [] thoseGuys);

I step into ConvertHodgmanKylotanjbadamsfrobRavyneApochPiQSeanMiddleditchrip_offspaceratLightness1024_to_String() and on the C side it works fine.  It generates the output I want as a local so I can verify it and then discards it to avoid leaking.

I need it to return the generated string to C#.  Let’s say I want this to be the final C function:
 


int __stdcall ConvertHodgmanKylotanjbadamsfrobRavyneApochPiQSeanMiddleditchrip_offspaceratLightness1024_to_String(const char * thoseGuys, char * retVal, int &retLen);

How should I allocate it in C and how do I tell C# how to treat it as a parameter and how do I deallocate it after?


L. Spiro

I restore Nintendo 64 video-game OST’s into HD! https://www.youtube.com/channel/UCCtX_wedtZ5BoyQBXEhnVZw/playlists?view=1&sort=lad&flow=grid

Advertisement

Top answer here has some useful options to explore.

Wielder of the Sacred Wands
[Work - ArenaNet] [Epoch Language] [Scribblings]

If you are actually working with strings in particular, the C# side should use string for the C#->C param, then probably a StringBuilder with a predefined size for the C->C# "return value". You should probably make sure to select reasonable MarshalAs options anyway though.

Another way, that is possibly easier (but uglier) is to just use unsafe and IntPtr. Depending on your target that may cause problem due to the unsafe requirement. In this case, make sure to look up the fixed and unsafe keywords. If I understand things correctly, this will bypass most if not all marshalling and copying, but is less clean and "safe".

If you are working with arrays of primitives, you may need to tag the parameters with [In] and/or [Out] depending on the direction of the data. That also applies to ref parameters.

If you need to pass structs, make sure you use struct in C# and make sure to tag any non-primitive types (including arrays of primitive types) with reasonable MarshalAs values and make sure to tag the entire struct with [StructLayout(LayoutKind.Sequential)] (or whatever the explicit version is in which case you should tag each field with their absolute offset as well). Let it be known that I have seen unexpected or problematic behaviour when trying to pass structs with non-primitive members though. In one of those cases I had much better luck serializing the data to a byte array and pass it as an IntPtr instead of trying to get the marshalling to work correctly, but that particular case was a rather complicated struct in an API beyond my control with nested structs, strings and other fixed size arrays of nested structs and stuff, so YMMW.

[Edit]

I just realized I missed a part of your post where you wanted to allocate the return value in C. I'm not sure that's what you actually want given your C signature which looks more like you want to pass in a preallocated buffer (in which case you want to use StringBuilder) but if you actually want to allocate it in your C function, I would return it as a pointer and use it with IntPtr in C#, then define some form of FreeString() function in your C dll and pass that IntPtr back once you've manually copied the data into a C# array/struct/string/whatever. Which is how I would do it in C if I was designing an API, even if it was not strictly meant to be used from C#.

I was idiot and made error in my C prototype.  The C function would have needed “char * &retVal” so it could return its own allocation to C#.

As you can guess, converting all those people to strings can give wildly different return sizes.  I expect ranges from a few bytes to a few megabytes.  ApochPiQ alone is probably 3 megabytes of a string.  So it is not something I can allocate ahead of time.

But I think the use of BSTR mentioned in the link ApochPiQ posted should work.  I am fine with returning a buffer as long as it can be cleaned up by C# properly.  So they call it “PInvoke” eh?  That should be a handy term to use in my future searches.


L. Spiro

I restore Nintendo 64 video-game OST’s into HD! https://www.youtube.com/channel/UCCtX_wedtZ5BoyQBXEhnVZw/playlists?view=1&sort=lad&flow=grid

I've never used BSTR myself, but personally I think it looks rather scary. It will probably not work (or work strange) with other data types and I'd also be very wary of cross platform compatibility with this. On the other hand, if it's not for a long-term cross platform project then the easiest and simplest way is to be preferred I suppose. 

Something like this is how I would do it:


    static class PInvokeThing
    {
        [DllImport("Important.dll", CharSet = CharSet.Ansi, CallingConvention = CallingConvention.StdCall)]
        private static unsafe extern int ConvertHodgmanKylotanjbadamsfrobRavyneApochPiQSeanMiddleditchrip_offspaceratLightness1024_to_String(string stuff, out byte* outPtr, out int outLen);

        [DllImport("Important.dll", CharSet = CharSet.Ansi, CallingConvention = CallingConvention.StdCall)]
        private static unsafe extern int FreeBuffer(byte* ptr);


        public static string ConvertTheString(string input)
        {
            unsafe
            {
                byte* ptr;
                int len;
                var result = ConvertHodgmanKylotanjbadamsfrobRavyneApochPiQSeanMiddleditchrip_offspaceratLightness1024_to_String(input, out ptr, out len);

                // check result

                var barr = new byte[len];

                for (int i = 0; i < len; i++)
                    barr[i] = ptr[i];

                FreeBuffer(ptr); // probably check this too

                return Encoding.Default.GetString(barr);
            }
        }
    }

This was written mostly from my head and may contain minor mistakes, but the idea should be usable for most types of data etc.

The other option is not to use pinvoke, but C++/CLI. 

It depends on how complex your interaction at the boundary is. 

if you think programming is like sex, you probably haven't done much of either.-------------- - capn_midnight

I did some test interaction functions from C# DLL into my C++ engine structure that lets the natie side create and manage those data while the managed side has read/write access to it so it is possible to pass for example an arrays native pointer from managed side to the native side for operating.

My Array.cs class looks like this


using System;
using System.Collections.Generic;
using System.Runtime.InteropServices;

namespace Bridge
{
    public interface IArrayInternal
    {
        IntPtr Handle { get; set; }
        Allocator.IAllocator GetAllocator();

        int GetLength();

        object GetValue(int index);
        void SetValue(int index, object value);

        /// <summary>
        /// Resizes this array. Any data inside might get lost in order
        /// ti achieve the targeted size
        /// </summary>
        void Resize(int size);
        void Dispose();
    }
    public class ByteArrayInternal : Interopt.NativeMngClass, IArrayInternal
    {
        public new IntPtr Handle
        {
            get { return ptr; }
            set { ptr = value; }
        }
        public virtual int GetLength()
        {
            if (ptr == IntPtr.Zero) return -1;
            else return (int)csGetArraySizeByte(ptr);
        }

        public ByteArrayInternal(IntPtr ptr, Allocator.IAllocator allocator = null)
            : base(ptr, allocator, true)
        { }
        public ByteArrayInternal(UInt32 size, Allocator.IAllocator allocator = null)
            : base(IntPtr.Zero, allocator, true)
        {
            if((ptr = csCreateArrayByte(GetAllocator().Handle)) != IntPtr.Zero)
                csArrayResizeByte(ptr, size);
        }

        public object GetValue(int index)
        {
            if (ptr == IntPtr.Zero) return 0;
            else return csGetArrayValueByte(ptr, (UInt32)index);
        }
        public void SetValue(int index, object value)
        {
            if (ptr != IntPtr.Zero)
                csSetArrayValueByte(ptr, (UInt32)index, (byte)value);
        }

        public virtual void Resize(int size)
        {
            if (ptr == IntPtr.Zero) return;
            else csArrayResizeByte(ptr, (UInt32)size);
        }

        public override void Dispose()
        {
            if (ptr != IntPtr.Zero) csDeleteArrayByte(ptr);
            ptr = IntPtr.Zero;

            Unregister();
        }

        [DllImport("CSharp.dll", EntryPoint = "csCreateArrayByte", CallingConvention = CallingConvention.Cdecl)]
        private static extern IntPtr csCreateArrayByte(IntPtr allocator);

        [DllImport("CSharp.dll", EntryPoint = "csGetArraySizeByte", CallingConvention = CallingConvention.Cdecl)]
        private static extern UInt32 csGetArraySizeByte(IntPtr instance);

        [DllImport("CSharp.dll", EntryPoint = "csGetArrayValueByte", CallingConvention = CallingConvention.Cdecl)]
        private static extern byte csGetArrayValueByte(IntPtr instance, UInt32 index);

        [DllImport("CSharp.dll", EntryPoint = "csSetArrayValueByte", CallingConvention = CallingConvention.Cdecl)]
        private static extern void csSetArrayValueByte(IntPtr instance, UInt32 index, byte value);

        [DllImport("CSharp.dll", EntryPoint = "csArrayResizeByte", CallingConvention = CallingConvention.Cdecl)]
        private static extern void csArrayResizeByte(IntPtr instance, UInt32 size);

        [DllImport("CSharp.dll", EntryPoint = "csDeleteArrayByte", CallingConvention = CallingConvention.Cdecl)]
        private static extern void csDeleteArrayByte(IntPtr instance);
    }

    public class Array<T> : Interopt.INativeClass, Interopt.INativeMngClass, IDisposable
    {
        protected IArrayInternal handler;

        public IntPtr Handle 
        {
            get 
            {
                if (handler == null) return IntPtr.Zero;
                else return handler.Handle; 
            }
            protected set
            {
                if (handler == null) return;
                else handler.Handle = value;
            }
        }
        public Allocator.IAllocator GetAllocator()
        {
            if (handler == null) return null;
            else return handler.GetAllocator();
        }

        public virtual int Length
        {
            get 
            {
                if (handler == null) return 0;
                else return handler.GetLength(); 
            }
        }

        public Array()
        { }
        public Array(IntPtr ptr, Allocator.IAllocator allocator = null)
        {
            switch (Type.GetTypeCode(typeof(T)))
            {
                case TypeCode.Byte: handler = new ByteArrayInternal(ptr, allocator); break;
                default: throw new ArgumentOutOfRangeException();
            }
        }
        public Array(UInt32 size, Allocator.IAllocator allocator = null)
        {
            switch (Type.GetTypeCode(typeof(T)))
            {
                case TypeCode.Byte: handler = new ByteArrayInternal(size, allocator); break;
                default: throw new ArgumentOutOfRangeException();
            }
        }
        ~Array()
        {
            Dispose();
        }

        public T this[int index]
        {
            get 
            {
                return (T)Convert.ChangeType(handler.GetValue(index), typeof(T)); 
            }
            set 
            {
                if (handler != null)
                    handler.SetValue(index, value); 
            }
        }

        public virtual void Resize(int size)
        {
            if(handler != null)
                handler.Resize(size);
        }
        public virtual void Dispose()
        {
            if (handler != null)
                handler.Dispose();
        }

        public override string ToString()
        {
            return string.Format("{0}[{1}]", typeof(T).Name, Length);
        }
    }
}

Used this schema for having some of my C# tools to access engine native functions

PInvoke is one term, data marshalling is another. 

PInvoke.net is a good resource, but even so, the effort involved is still an annoyance. In working with C you still need to operate in terms of sources and sinks, and when the C side function is allocating, you'll need a corresponding C side function to release it.

This topic is closed to new replies.

Advertisement