[.net] Program crashes when returning a struct from a DllImport function?

Started by
6 comments, last by TheBluMage 16 years, 5 months ago
I seem to run into all the weird problems. Anyway, I'll cut to the chase. Here's the C struct that the function is supposed to return:
struct AVS_Value {
  short type;  // 'a'rray, 'c'lip, 'b'ool, 'i'nt, 'f'loat, 's'tring, 'v'oid, or 'l'ong
               // for some function e'rror
  short array_size;
  union {
    void * clip; // do not use directly, use avs_take_clip
    char boolean;
    int integer;
    float floating_pt;
    const char * string;
    const AVS_Value * array;
  } d;
};

And here's the signature of the function that returns it:
AVSC_API(AVS_Value) avs_invoke(AVS_ScriptEnvironment *, const char * name, AVS_Value args, const char** arg_names);

AVSC_API basically just says that the function using the stdcall convention and that it's a dllexport function. Here's my C# translation of the C struct:
[StructLayout(LayoutKind.Explicit)]
public struct SValue {

	[FieldOffset(0)] public char Type;
	[FieldOffset(2)] public short ArrayLength;
	[FieldOffset(4)] public IntPtr Data;
	[FieldOffset(4)] public int Integer;
	[FieldOffset(4)] public float Float;

}

When the pointer is used I do all the translations manually, since you can't have a string in the same place as a value type. Anyway, that works fine. Finally, here's the imported DLL function:
[DllImport("avisynth.dll", EntryPoint = "avs_invoke", CallingConvention = CallingConvention.StdCall)]
public static extern SValue _Invoke(IntPtr env, [MarshalAs(UnmanagedType.LPStr)] string name, SValue args, [MarshalAs(UnmanagedType.LPArray, ArraySubType = UnmanagedType.LPStr)] string[] argNames);

The problem is, calling this function in my program makes the application crash. (No exception, it just crashes. "AppName.exe has encountered a problem and needs to close... yada yada yada.") But, I can make it work if I have the imported function return a ulong (which is an 8-byte value type, just like my struct) instead of an SValue, like so:
[DllImport("avisynth.dll", EntryPoint = "avs_invoke", CallingConvention = CallingConvention.StdCall)]
public static extern ulong _Invoke(IntPtr env, [MarshalAs(UnmanagedType.LPStr)] string name, SValue args, [MarshalAs(UnmanagedType.LPArray, ArraySubType = UnmanagedType.LPStr)] string[] argNames);

When I do this, the function call executes just fine and it behaves just the way it's supposed to. I can cast the returned ulong to an SVideo and it works fine. But... why is there a problem returning a struct from the function?
Advertisement
My first guess is that it is actually returning a pointer to the struct and not the struct itself. If this is the case make the return type from the external method IntPtr and use Marshal.PtrToStructure() to convert it into your struct.

Hope this helps.
I suspected this might be the case, but if that were so, casting the ulong to the struct wouldn't work. I've also used the function in C and the documented signature is correct.
Very strange. So the ulong when cast to your struct contains the valid values that you expect? Perhaps it has something to do with the union in the returned struct (I mean it really should not care as long as the struct is the expected size) or the calling convention (have you tried cdecl -- I have had similar crashes in the past when the convention was just wrong -- mind you I was not trying to deal with a returned struct). Anyway this is well beyond my knowledge at this point. Sorry I could not be of more help.

one more thing: shouldn't you be specifying the size in your StructLayout?
Tried specifying the size and the other calling convention, but neither resolves the problem. And, yes, the returned ulong contains all the correct data and can be converted to an SValue like so:

SValue v = *((SValue*) &returnedLong);

I just wish this ugly cast wasn't needed and I can't come up with any reason why it is. Anyway, thanks for trying. :(
I finally figured it out! Or at least I've managed to make it work. Placing the MarshalAs(UnmanagedType.I2) attribute in front of the char in my struct makes it work. My best guess is that it was getting marshaled as though it was an ANSI char, one byte long. Previously, I had assumed that it would be marshaled as a wchar, since the default behavior for strings is to use that type of character, but I guess I was mistaken. (I'm still somewhat confused as to why I could pass the struct as input and have it work okay, but not have it return from the function, but I guess it doesn't really matter.)
Would CharSet=CharSet.Unicode solve that?
Whaddya know - that works too!

This topic is closed to new replies.

Advertisement