I have the following C++ function definition, which I am trying to call through PInvoke from managed code:
bool FooBar(SIZE_T* arg1);
My managed declaration looked as follows:
[DllImport("mydll", SetLastError=true, CharSet=CharSet.Unicode)]
private static extern bool FooBar(ref uint arg1);
Some of you may notice the same bug I eventually did. This is not 64bit portable. SIZE_T is of variable size (32-64 bit) as is the pointer to it. On the managed size the pointer correctly translates to 64bit, but the uint does not, and you can end up with trash in the upper bits of arg1. This was an especially persistent error since the trash was often just zeros :(
The only solution I have gotten to work is the following managed declaration:
[DllImport("mydll", SetLastError=true, CharSet=CharSet.Unicode)]
private static extern bool FooBar(ref IntPtr arg1);
This works of course because IntPtr can vary its size correctly. In my code I just treat the IntPtr as an integer, and it works, though it looks like an ugly hack. It seems to me there should be some way to specify this properly, perhaps using UnmanagedType.SysUInt, but I have been unable to come up with any other working solution.