Allocating unmanaged memory in managed .NET code

Question

Hi I have a unmanaged function that takes a chunk of memory allocated by malloc and deallocates it later on in async manner. I want to wrap it into managed wrapper. Is following code OK?

void managed_fx (byte data __gc[], size_t size)
{
    //  Pin the data
    byte __pin *pinned_data = &data [0];

    //  Copy data to the unmanaged buffer.
    void *unmanaged_data = malloc (size);
    memcpy (unmanaged_data, (byte*) pinned_data, size);

    //  Forward the call
    fx (unmanaged_data, size);
}

Answer 1

you need to test your malloc that it actually has allocated your pointer. Don't know C++/CLI so I don't know if your pin will work. Don't you need to use the Marshalling library to guarantee that the memory gets pinned while you copy it?

Answer 2

Read this for information on how to allocate unmanaged data. Take a look of Marshall class. Maybe there would be some method which can replace your managed_fx function.

Answer 3

You are

1. assuming that the data array is at least as big a size. This is a possible bug waiting to happen

2. Not checking the result of malloc()

3. You may not need to do this anyway, simply passing the pinned byte pointer should be fine (unless the called fx() function in some way changes the data in which case the copy is perhaps a requirement). Based on the memory being freed later it is likely that this is not a possibility though.

With that in mind here is a fixed up version

void managed_fx (byte data __gc[], size_t size)
{
    if (data.Length < size)
        throw gcnew System.IndexOutOfRangeException(
            "the unmanaged buffer is not big enough");
    //  Pin the data
    byte __pin *pinned_data = &data [0];

    //  Copy data to the unmanaged buffer.
    void *unmanaged_data = malloc (size);
    if (unmanaged_data == NULL) 
        throw gcnew OutOfMemoryException(
            "could not allocate memory for the unmanaged buffer");
    memcpy (unmanaged_data, (byte*) pinned_data, size);

    //  Forward the call
    fx (unamanged_data, size);
}

To answer the further questions:

The pinning is necessary to do the memcpy. It does not prevent the deletion of the memory by the GC runtime (well technically it does but so does simply holding of a reference to it) it prevents it moving in memory. Since memcpy is an unmanaged function - it cannot handle the pointers it is manipulating being moved about in memory. malloc is entirely unmanaged (though the runtime libraries are free to use reserved areas of the managed heap to do this).

There is no clear distinction in source C++/CLI as to whether 'code' is managed or unmanaged. On compilation some parts will be CIL intermediate code, others plain native code the important thing is whether the variables used are managed or unmanaged. A managed reference (anything with the ^) can only be manipulated/used by code which is itself managed. This means the runtime is free to alter the values of the underlying pointers used as it sees fit if it alters the location of the managed object in memory. It does this in a way that is safe to managed functions (they are paused while the world is changed around them). Since many useful functions exist with no knowledge of how to handle such managed references you may end up needing to take a simple pointer to them. Since this operation is unsafe (if the GC kicks in a moves the reference about) the pinning pointer exists to make this safe and simple to do since it both takes a pointer and for the life of that pointer prevents the runtime moving the pointed to variable.

Answer 4

Ok, let's make it even simpler:

void managed_fx (byte data __gc[])
{
     //  Pin the data
     byte __pin *pinned_data = &data [0];

     //  Copy data to the unmanaged buffer.
     void *unmanaged_data = malloc (data.Length);
     assert (unmanaged_data);
     memcpy (unmanaged_data, (byte*) pinned_data, data.Length);

     //  Forward the call
     fx (unamanged_data, data.Length);
 }

Still, I have couple of questions:

1. Is pinning necessary at all in this case? Can gc deallocate it before memcpy happens?
2. Does malloc allocate an unmanaged memory even though it's used in managed code?

Answer 5

My MC++ is a little rusty, but I think __pin pins the source variable until "pinned_data" goes out of scope (at least, that's what the equivalent pin_ptr in C++/CLI does). You should generally un-pin it as soon as possible, i.e. you shouldn't call fx in the same scope for performance reasons.

Is pinning necessary at all in this case?

Yes. You want to access managed memory from unmanaged code.

Can gc deallocate it before memcpy

No. There is a strong reference to data, so the gc won't collect it. It might however collect some other object, and move data in memory in the compacting step. Then malloc would access the wrong memory area. Pinning prevents this, at the cost of additional bookkeeping for the GC (which is why you should un-pin objects as soon as possible).

Does malloc allocate an unmanaged memory even though it's used in managed code?

Yes.