views:

310

answers:

5

Function signature:

char * errMessage(int err);

My code:

[DllImport("api.dll")]       
internal static extern char[] errMessage(int err);
...
char[] message = errMessage(err);

This returns an error:

Cannot marshal 'return value': Invalid managed/unmanaged type combination.

What am I doing wrong? Thanks for any help.

+3  A: 

try this:

[DllImport("api.dll")]
[return : MarshalAs(UnmanagedType.LPStr)]
internal static extern string errMessage(int err);
...
string message = errMessage(err);

I believe C# is smart enough to handle the pointer and return you a string.

Edit: Added the MarshalAs attribute

scottm
No, this causes it not to find the correct entry point.
IronicMuffin
Even with the marshal as, it loses the ability to find the entry point if it is changed to string.
IronicMuffin
This is the solution I would want. Do we know what we can do to make this work?
IronicMuffin
maybe try to declare the prototype as returning int?
scottm
A: 

Try using String in the managed side. you might as well set the CharSet to Ansi

Moshe Levi
CharSet probably is Ansi, but hopefully IronicMuffin knows what it is.
MarkJ
+4  A: 

See this question. To summary, the function should return an IntPtr and you have to use Marshal.PtrToString* to convert it to a managed String object.

cedrou
What's the best practice to free the space allocated for the string with this solution?
dtb
It depends on how the memory was allocated. If it was allocated with CoTaskMemAlloc, you should use Marshal.FreeCoTaskMem. Otherwise, don't free it, because it was certainly allocated with new/malloc in a heap which not belongs to your Assembly.
cedrou
+2  A: 

A simple and robust way is to allocate a buffer in C# in the form if a StringBuilder, pass it to the unmanaged code and fill it there.

Example:

C

#include <string.h>

int foo(char *buf, int n) {
   strncpy(buf, "Hello World", n);
   return 0;
}

C#

[DllImport("libfoo", EntryPoint = "foo")]
static extern int Foo(StringBuilder buffer, int capacity);

static void Main()
{
    StringBuilder sb = new StringBuilder(100);
    Foo(sb, sb.Capacity);
    Console.WriteLine(sb.ToString());
}

Test:

Hello World
dtb
This would be great but I don't have access to the unmanaged source.
IronicMuffin
+2  A: 

It is a horrible function signature, there's no way to guess how the string was allocated. Nor can you deallocate the memory for the string. If you declare the return type as "string" in the declaration then the P/Invoke marshaller will call CoTaskMemFree() on the pointer. That is very unlikely to be appropriate. It will silently fail in XP but crash your program in Vista and Win7.

You can't even reliably call the function in an unmanaged program. The odds that you'd use the correct version of free() are pretty slim. All you can do is declare it as IntPtr and marshal the return value yourself with Marshal.PtrToStringAnsi(). Be sure to write a test program that does so a million times while you observe it in Taskmgr.exe. If the VM size for the program grows without bound, you have a memory leak you cannot plug.

Hans Passant