bool versus BOOLEAN in managed prototypes

Question

I'm trying to create a managed prototype in C# for the [CreateSymbolicLink][1] API function. The prototype in WinBase.h is:

BOOLEAN APIENTRY CreateSymbolicLink (
    __in LPCWSTR lpSymlinkFileName,
    __in LPCWSTR lpTargetFileName,
    __in DWORD dwFlags
    );

And BOOLEAN is defined as a BYTE in WinNT.h. Fine. So my managed prototype should be:

[DllImport("kernel32.dll", SetLastError = true)]
static extern bool CreateSymbolicLink(string SymlinkFileName, string TargetFileName, UInt32 Flags);

At least, I would think so. bool is just an alias for System.Boolean, a one-byte value. But it doesn't seem to work.

I execute this code:

bool retval = CreateSymbolicLink(LinkFilename, TargetFilename, 0);

It returns true. But the symbolic link isn't created. The reason it's not created is that I'm not running with elevated privileges. GetLastError returns 1314, which according to WinError.h means that I don't have the required permission. As expected. But why is my return value true?

Curiously, if I change the managed prototype to:

static extern byte CreateSymbolicLink(string SymlinkFileName, string TargetFileName, UInt32 Flags);

and my code to:

byte retval = CreateSymbolicLink(LinkFilename, TargetFilename, 0);

Then I get the expected result: retval contains 0, meaning that the function failed.

I guess my bigger question is why doesn't bool work for BOOLEAN return values?

Answer 1

Mark the return value with an appropriate MarshalAs attribute:

[DllImport("kernel32.dll", SetLastError = true)]
[return: MarshalAs(UnmanagedType.U1)]
static extern bool CreateSymbolicLink(string SymlinkFileName, 
    string TargetFileName, UInt32 Flags);

The default marshalling for bool from native to managed code is 4 bytes--you probably got true back for your original bool because one of the wrongly-marshalled stack bytes was nonzero. (I'm guessing on that last part.)

Answer 2

PInvoke with boolean types can produce unexpected results due to the differences between native and managed bools. I wrote a blog article on this subject detailing the differences and correct ways to marshal the various types

• http://blogs.msdn.com/jaredpar/archive/2008/10/14/pinvoke-and-bool-or-should-i-say-bool.aspx

Ben M has the correct answer in this case.