casting BSTR as char* in a dll; different results depnding on VB/C# caller.

Question

I have a dll function that takes BSTR parameters. These are casted as char* before being used for other things.

When the dll is called from VB code this works fine. However, when it is called from C# code, only the first character is pointed to.

Both of these are excel addIns for Pre-2007 and 2007+ versions of Office, which call into a faster C++ AddIn. They actually call it directly, not through Excel.

The VB function declaration looks like this:

Private Declare Function Test Lib "ExcelAddIn.xll" (ByVal param As String) As String

The C# function declaration looks like this:

[DllImport("ExcelAddIn.xll", CharSet=CharSet.Ansi)]
[return:MarshalAs(UnmanagedType.BStr)]
private static extern string Test([MarshalAs(UnmanagedType.BStr)] string param);

When debugging the dll and watching the input BSTR values, they appear to be correct from both; just the C# one only casts the first character.

Charset=CharSet.Unicode makes no difference.

Any ideas anyone?

[edit] Sorry, should probably post some C++ code.

BSTR __stdcall Test(BSTR param)
{
char* Param= "";

if(param!= NULL)
    Param= (char*)param;

    return OtherClass.DoOtherStuff(Param);
}

Answer 1

BSTRs are Unicode strings. As you're seeing, if you try and use a LE Unicode string as ANSI you'll only get the first letter (if it's ASCII, etc.)

ATL / MFC's CComBSTR class might help: you can attach the BSTR you receive and I think it'll handle single-byte conversion for you. It may also be safer to use this if you want Unicode strings internally - I don't think BSTRs are guaranteed to be zero-terminated (they have a length at index -1).[edit] corrected by dkackman below, thanks.

Your best solution really would be to convert OtherClass to use Unicode strings not MBCS, or if it's only used in a COM context you could convert it to use BSTRs even. Your program will then better support internationalisation expanding your markets, etc. However this isn't always a simple conversion.

Answer 2

The reason why has to do with the way you are marshalling the data.

The VB declaration contains no annotations on the string parameter and hence it will marshal as a normal char* parameter. The CLR can do no type verification for pinvoke and essentially puts a char* in a slot which expects a BSTR. Your code though immediately casts it to a char* and it fixes the marshalling problem.

In the C# example though you explicitly said to marshal the string as a BSTR. A BSTR is actually a string which uses wchar under the hood. By doing a simple cast to a char* you are essentially casting a wchar* to a char* which explains why you only see the first character.

The easiest solution is to just have the Test method take a char* parameter and remove the marshalling annotation on the C# version.