P/Invoke throw System.ExecutionEngineException

Question

I have a closed source unmanaged DLL coded in C++ that I wanted to use in a C# solution so I created a wrapper managed DLL that use P/Invoke to call the closed source DLL function. That works pretty well for no param function and int variables. However I get a System.ExecutionEngineException when running a more complex function that take an array of struct as parameter which contains array of char for strings. Here is what I had:

[StructLayout(LayoutKind.Sequential)]
public struct Target
{
    public int targetID;

    public string Label;
}

[DllImport("tyrfde.dll", EntryPoint = "tyrfdeGetTarget")]
public static extern int GetTarget(ref Target[] targets);

Below is the information I have from the header file of the DLL:

#define TARGET_LBL_SIZE   (256l)

typedef struct _tyrfdeTarget
{
    TInt32 TargetID;                   // integer signed 32bits
    TCharA Label[TARGET_LBL_SIZE];     // caracter
} tyrfdeTarget;

TInt32 __stdcall tyrfdeGetTargets(tyrfdeTarget* pTargets);

Not quite sure why the array size is specified as long but anyway SizeConst only take int. After some search here is what I tried to fix.

[StructLayout(LayoutKind.Sequential, Size = 260), Serializable]
public struct Target
{
    public int targetID;

    [MarshalAsAttribute(UnmanagedType.ByValTStr, SizeConst = 256)]
    public string Label;
}

[DllImport("tyrfde.dll", EntryPoint = "tyrfdeGetTargets")]
public static extern int GetTarget(ref Target[] targets);

But I still have the problem. I have read that this exception can throw if the functions clear part of the memory used by CLR. Unfortunately I cannot verify that. Is there something in my code that is obviously wrong and could cause the problem?

Answer 1

Hm, I think your problem is with the ref Target[] targets parameter. AFAIR this is a reference to a reference, which is probably not what you actually want.

I'd try this:

[DllImport("tyrfde.dll", EntryPoint = "tyrfdeGetTargets")]
public static extern int GetTarget([Out, MarshalAs(UnmanagedType.LPArray)] Target[] targets);

Maybe this article helps you to find the correct declaration.

Note that the size of the array is not clear here, usually in such cases there is also a ref int length parameter, which can then be referenced in the MarshalAs attribute via the SizeParameterIndex property.

Answer 2

1) Are you sure that TCharA is 16-bit? Otherwise I think you should also specify whar CharSet to use.

2) Writing this kind of wrappers is way way simpler in C++/CLI.