Is there an alternative for StructLayout "Pack" attribute in Compact Framework?

Question

I would like to do the following:

  [StructLayout(LayoutKind.Sequential, Pack = 1)]
  public struct SomeStruct
  {
     public byte  SomeByte;
     public int   SomeInt;
     public short SomeShort;
     public byte  SomeByte2;
  }

Is there an alternative since Pack is not supported in the compact framework?

Update: Explicitly setting up the structure and giving FieldOffset for each does not work either as it does not affect how the struct is packed

Update2: If you try the following, the CF program wont even run because of how the structure is packed:

[StructLayout(LayoutKind.Explicit, Size=8)]
public struct SomeStruct
{
   [FieldOffset(0)]
   public byte SomeByte;
   [FieldOffset(1)]
   public int SomeInt;
   [FieldOffset(5)]
   public short SomeShort;
   [FieldOffset(7)]
   public byte SomeByte2;
}

I know it seems hard to believe, but if you try it you will see. Add it to a CF project and try to run it and you will get a TypeLoadException. Changing the offsets to 0,4,8,10 respectively and it will work (but the size ends up being 12).

I was hoping maybe someone had a solution using reflection maybe to marshal the size of each of the field types individually (something involving recursion to handle structs within structs or arrays of types).

Answer 1

You need to post a more relevant example. Setting packing on that struct would have no effect anyway.

My bet is that you need to use LaoutKind.Explicit and then give the offsets for each member. It's way better than messing with the packing anyway, because it's way more obvious to someone looking at the code that the original developer explicitly meant for things to be unaligned.

Something along these lines:

[StructLayout(LayoutKind.Explicit)]
struct Foo
{
    [FieldOffset(0)]
    byte a;
    [FieldOffset(1)]
    uint b;
}

Answer 2

LayoutKind.Explicit would be your best bet for defining a specific memory layout. However, do not use LayoutKind.Explicit for structures that contain pointer-sized values such as true pointers, operating system handles or IntPtrs; this is just asking for mysterious trouble at runtime on random platforms.

In particular, LayoutKind.Explicit is a poor substitute for anonymous unions. If your target structure contains an anonymous union, convert it to a named union; you can safely represent a named union as a struct with LayoutKind.Explicit where all offsets are 0.

Answer 3

The simplest method of dealing with this type of problem is along the same lines as you might for a bit field, simply pack your data into a private member (or members if it is large) of the appropriate data type and then present public properties that unpack the data for you. The unpacking operations are extremely fast and will have little impact on performance. For your particular type the following is probably what you want:

public struct SomeStruct
{
    private long data;

    public byte SomeByte { get { return (byte)(data & 0x0FF); } }
    public int SomeInt { get { return (int)((data & 0xFFFFFFFF00) << 8); } }
    public short SomeShort { get { return (short)((data & 0xFFFF0000000000) << 40); } }
    public byte SomeByte2 { get { return (byte)((data & unchecked((long)0xFF00000000000000)) << 56); } }
}

For some structures even this method is not workable due to the unfortunate way a structure has been defined. In those cases you will generally have to use a byte array as a blob of data from which the elements can be unpacked.

EDIT: To expand on what I mean about structs that can't be handled using this simple method. When you can't do simple packing/unpacking like this you need to manually marshal the irregular struct . This can be done using manual methods at the point you call the pInvoked API or by using a custom marshaler. The following is an example of a custom marhsaler that can be easily adapted to on the spot manual marshaling.

using System.Runtime.InteropServices;
using System.Threading;

public class Sample
{
    [DllImport("sample.dll")]
    public static extern void TestDataMethod([MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef = typeof(TestDataMarshaler))] TestDataStruct pData);
}

public class TestDataStruct
{
    public byte data1;
    public int data2;
    public byte[] data3 = new byte[7];
    public long data4;
    public byte data5;
}

public class TestDataMarshaler : ICustomMarshaler
{
    //thread static since this could be called on 
    //multiple threads at the same time.
    [ThreadStatic()]
    private static TestDataStruct m_MarshaledInstance;

    private static ICustomMarshaler m_Instance = new TestDataMarshaler();

    public static ICustomFormatter GetInstance(string cookie)
    {
        return m_Instance;
    }

    #region ICustomMarshaler Members

    public void CleanUpManagedData(object ManagedObj)
    {
        //nothing to do.
    }

    public void CleanUpNativeData(IntPtr pNativeData)
    {
        Marshal.FreeHGlobal(pNativeData);
    }

    public int GetNativeDataSize()
    {
        return 21;
    }

    public IntPtr MarshalManagedToNative(object ManagedObj)
    {
        m_MarshaledInstance = (TestDataStruct)ManagedObj;
        IntPtr nativeData = Marshal.AllocHGlobal(GetNativeDataSize());

        if (m_MarshaledInstance != null)
        {
            unsafe //unsafe is simpler but can easily be done without unsafe if necessary
            {
                byte* pData = (byte*)nativeData;
                *pData = m_MarshaledInstance.data1;
                *(int*)(pData + 1) = m_MarshaledInstance.data2;
                Marshal.Copy(m_MarshaledInstance.data3, 0, (IntPtr)(pData + 5), 7);
                *(long*)(pData + 12) = m_MarshaledInstance.data4;
                *(pData + 20) = m_MarshaledInstance.data5;
            }
        }
        return nativeData;
    }

    public object MarshalNativeToManaged(IntPtr pNativeData)
    {
        TestDataStruct data = m_MarshaledInstance;
        m_MarshaledInstance = null; //clear out TLS for next call.

        if (data == null) data = new TestDataStruct(); //if no in object then return a new one

        unsafe //unsafe is simpler but can easily be done without unsafe if necessary
        {
            byte* pData = (byte*)pNativeData;
            data.data1 = *pData;
            data.data2 = *(int*)(pData + 1);
            Marshal.Copy((IntPtr)(pData + 5), data.data3, 0, 7);
            data.data4 = *(long*)(pData + 12);
            data.data5 = *(pData + 20);
        }
        return data;
    }

    #endregion
}

In the case of arrays of these structs you can't use custom marshaling unless the array size is fixed but it is relatively easy to manually marshal the array data as a whole using the same techniques.

Answer 4

LayoutKind.Explicit and FieldOffsetAttribute will allow you to do anything you could do with the Pack property. These explicit layout attributes allow you to specify the exact byte position of each field in the struct (relative to the beginning of the struct's range of memory). The Pack property is used by the runtime to help determine the exact position of each field when using a sequential layout. The pack property has no other effect, so using explicit layout allows you to emulate the exact same behavior, albeit a bit more verbosely. If you don't think this solves your problem, perhaps you could post a bit more information about what you're trying to do or why you think you need to use the Pack property.

Edit: I just noticed the additional comment about trying to get the entire structure's size to 8 bytes. Have you tried using the StructLayoutAttribute.Size property? Unlike Pack, it is available in the Compact Framework.

Answer 5

This probably isn't the type of answer you're looking for, but I'll post it anyway for the hell of it:

public struct SomeStruct
{
    public byte SomeByte;
    public int SomeInt;
    public short SomeShort;
    public byte SomeByte2;


    public byte[] APIStruct
    {
        get
        {
            byte[] output = new byte[8];
            output[0] = this.SomeByte;
            Array.Copy(BitConverter.GetBytes(this.SomeInt), 0,
                output, 1, 4);
            Array.Copy(BitConverter.GetBytes(this.SomeShort), 0,
                output, 5, 2);
            output[7] = this.SomeByte2;
            return output;
        }
        set
        {
            byte[] input = value;
            this.SomeByte = input[0];
            this.SomeInt = BitConverter.ToInt32(input, 1);
            this.SomeShort = BitConverter.ToInt16(input, 5);
            this.SomeByte2 = input[7];
        }
    }
}

Basically it does the packing/unpacking itself in the APIStruct property.

Answer 6

Do you absolutely require that specific layout or is it acceptable to simply make the size 8?

I ask this because the lay out as follows

[StructLayout(LayoutKind.Explicit, Size=8)]
public struct SomeStruct
{
   [FieldOffset(0)]
   public byte SomeByte;
   [FieldOffset(1)]
   public int SomeInt;
   [FieldOffset(5)]
   public short SomeShort;
   [FieldOffset(7)]
   public byte SomeByte2;
}

Has non word aligned fields which may be what is causing your problem.

If you can 'rearrange' things then this might work for you:

[StructLayout(LayoutKind.Explicit, Size=8)]
public struct SomeStruct
{
   [FieldOffset(0)]
   public byte SomeByte;
   [FieldOffset(1)]
   public byte SomeByte2;
   [FieldOffset(2)]
   public short SomeShort;
   [FieldOffset(4)]
   public int SomeInt;
}

When I test with this on the emulator it works fine.

Obviously unless you are willing to allow the rearrangement there's nothing you can do.

This answer and this old article would strongly indicate that you must at a minimum align your structs on multiples of their size (I tried with an int aligned on offset 2 and this also triggered the error)

Given your need to interoperate with externally defined data then the following is likely your easiest solution:

[StructLayout(LayoutKind.Explicit, Size=8)]
public struct SomeStruct
{ 
   [FieldOffset(0)] private byte b0;
   [FieldOffset(1)] private byte b1;
   [FieldOffset(2)] private byte b2;
   [FieldOffset(3)] private byte b3;
   [FieldOffset(4)] private byte b4;
   [FieldOffset(5)] private byte b5;
   [FieldOffset(6)] private byte b6;
   [FieldOffset(7)] private byte b7;

   // not thread safe - alter accordingly if that is a requirement
   private readonly static byte[] scratch = new byte[4];       

   public byte SomeByte 
   { 
       get { return b0; }
       set { b0 = value; }
   }

   public int SomeInt
   {
       get 
       { 
           // get the right endianess for your system this is just an example!
           scratch[0] = b1;
           scratch[1] = b2;
           scratch[2] = b3;
           scratch[3] = b4;
           return BitConverter.ToInt32(scratch, 0);
       }
   }

   public short SomeShort
   {
        get 
        { 
            // get the right endianess for your system this is just an example!
            scratch[0] = b5;
            scratch[1] = b6;
            return BitConverter.ToInt16(scratch, 0);
        }
    }

    public byte SomeByte2 
    { 
        get { return b7; }
        set { b7 = value; }
    }
}

Answer 7

I think one should take Stephen Martin's answer, make it accept a T, and use reflection to generically implement the MarshalManagedToNative and MarshalNativeToManaged methods. Then, you'll have a custom packed struct marshaler that will work for any type of struct.

Here's the code (compiled but untested):

using System;
using System.Threading;
using System.Reflection;
using System.Runtime.InteropServices;

namespace System.Runtime.InteropServices
{
    public class PinnedObject : IDisposable
    {
        private GCHandle gcHandle = new GCHandle();
        public PinnedObject(object o)
        {
            gcHandle = GCHandle.Alloc(o, GCHandleType.Pinned);
        }

        public unsafe static implicit operator byte*(PinnedObject po)
        {
            return (byte*)po.gcHandle.AddrOfPinnedObject();
        }

        #region IDisposable Members
        public void Dispose()
        {
            if (gcHandle.IsAllocated)
            {
                gcHandle.Free();
            }
        }
        #endregion
    }

    public class PackedStructMarshaler<T> : ICustomMarshaler where T : struct
    {
        private static ICustomMarshaler m_instance = new PackedStructMarshaler<T>();

        public static ICustomMarshaler GetInstance()
        {
            return m_instance;
        }

        private void ForEachField(Action<FieldInfo> action)
        {
            foreach (var fi in typeof(T).GetFields(BindingFlags.Public | BindingFlags.NonPublic))
            {
                // System.Diagnostics.Debug.Assert(fi.IsValueType);
                action(fi);
            }
        }

        private unsafe void MemCpy(byte* dst, byte* src, int numBytes)
        {
            for (int i = 0; i < numBytes; i++)
            {
                dst[i] = src[i];
            }
        }

        #region ICustomMarshaler Members
        public void CleanUpManagedData(object ManagedObj)
        {
        }

        public void CleanUpNativeData(IntPtr pNativeData)
        {
            Marshal.FreeHGlobal(pNativeData);
        }

        public int GetNativeDataSize()
        {
            unsafe
            {
                int ret = 0;
                ForEachField(
                    (FieldInfo fi) =>
                    {
                        Type ft = fi.FieldType;
                        ret += Marshal.SizeOf(ft);
                    });
                return ret;
            }
        }

        private object m_marshaledObj = null;

        public unsafe IntPtr MarshalManagedToNative(object obj)
        {
            IntPtr nativeData = (IntPtr)0;

            if (obj != null)
            {
                if (m_marshaledObj != null)
                    throw new ApplicationException("This instance has already marshaled a managed type");

                m_marshaledObj = obj;

                nativeData = Marshal.AllocHGlobal(GetNativeDataSize());
                byte* pData = (byte*)nativeData;
                int offset = 0;

                ForEachField(
                    (FieldInfo fi) =>
                    {
                        int size = Marshal.SizeOf(fi.FieldType);
                        using (PinnedObject po = new PinnedObject(fi.GetValue(obj)))
                        {
                            MemCpy(pData + offset, po, size);
                        }
                        offset += size;
                    });
            }

            return nativeData;
        }

        public object MarshalNativeToManaged(IntPtr pNativeData)
        {
            if (m_marshaledObj != null)
                m_marshaledObj = null;

            unsafe
            {
                byte* pData = (byte*)pNativeData;
                int offset = 0;

                object res = new T();

                ForEachField(
                    (FieldInfo fi) =>
                    {
                        int size = Marshal.SizeOf(fi.FieldType);
                        fi.SetValue(res, (object)(*((byte*)(pData + offset))));
                        offset += size;
                    });

                return res;
            }
        }

        #endregion
    }
}