No it isn't. The internal layout of a class or struct is undiscoverable. Marshaling is required, guided by a [StructLayout], to convert that undocumented layout to a known one. The JIT compiler readily takes advantage of this, it re-orders the fields in a struct for example to get them properly aligned and require the minimum of storage. This defeats any tricks that messes with unmanaged pointers. The simple value types behave predictably, but they are already well covered by BitConverter. Structures are your nemesis.
This is one reason why it took so long for memory-mapped files to be supported by the .NET framework. But they'll be available in .NET 4.0, you could take advantage of the MemoryMappedViewAccessor class. It still uses marshaling, it is hidden under the floor mat.