Can address space be recycled for multiple calls to MapViewOfFileEx without chance of failure?

Question

Consider a complex, memory hungry, multi threaded application running within a 32bit address space on windows XP.

Certain operations require n large buffers of fixed size, where only one buffer needs to be accessed at a time.

The application uses a pattern where some address space the size of one buffer is reserved early and is used to contain the currently needed buffer.

This follows the sequence: (initial run) VirtualAlloc -> VirtualFree -> MapViewOfFileEx (buffer changes) UnMapViewOfFile -> MapViewOfFileEx

Here the pointer to the buffer location is provided by the call to VirtualAlloc and then that same location is used on each call to MapViewOfFileEx.

The problem is that windows does not (as far as I know) provide any handshake type operation for passing the memory space between the different users.

Therefore there is a small opportunity (at each -> in my above sequence) where the memory is not locked and another thread can jump in and perform an allocation within the buffer.

The next call to MapViewOfFileEx is broken and the system can no longer guarantee that there will be a big enough space in the address space for a buffer.

Obviously refactoring to use smaller buffers reduces the rate of failures to reallocate space.

Some use of HeapLock has had some success but this still has issues - something still manages to steal some memory from within the address space. (We tried Calling GetProcessHeaps then using HeapLock to lock all of the heaps)

What I'd like to know is there anyway to lock a specific block of address space that is compatible with MapViewOfFileEx?

Edit: I should add that ultimately this code lives in a library that gets called by an application outside of my control

Answer 1

Have you looked at creating your own private heap via HeapCreate? You could set the heap to your desired buffer size. The only remaining problem is then how to get MapViewOfFileto use your private heap instead of the default heap.

I'd assume that MapViewOfFile internally calls GetProcessHeap to get the default heap and then it requests a contiguous block of memory. You can surround the call to MapViewOfFile with a detour, i.e., you rewire the GetProcessHeap call by overwriting the method in memory effectively inserting a jump to your own code which can return your private heap.

Microsoft has published the Detour Library that I'm not directly familiar with however. I know that detouring is surprisingly common. Security software, virus scanners etc all use such frameworks. It's not pretty, but may work:

HANDLE g_hndPrivateHeap;

HANDLE WINAPI GetProcessHeapImpl() {
    return g_hndPrivateHeap;
}    


struct SDetourGetProcessHeap { // object for exception safety 
   SDetourGetProcessHeap() {
       // put detour in place
   }

   ~SDetourGetProcessHeap() {
       // remove detour again
   }
};


void MapFile() {
    g_hndPrivateHeap = HeapCreate( ... );

    {
        SDetourGetProcessHeap d;
        MapViewOfFile(...);
    }
}

These may also help:

http://stackoverflow.com/questions/60641/how-to-replace-winapi-functions-calls-in-the-ms-vc-project-with-my-own-implemen

http://stackoverflow.com/questions/873658/how-can-i-hook-windows-functions-in-c-c

http://research.microsoft.com/pubs/68568/huntusenixnt99.pdf

Answer 2

Imagine if I came to you with a piece of code like this:

void *foo;

foo = malloc(n);
if (foo)
   free(foo);
foo = malloc(n);

Then I came to you and said, help! foo does not have the same address on the second allocation!

I'd be crazy, right?

It seems to me like you've already demonstrated clear knowledge of why this doesn't work. There's a reason that the documention for any API that takes an explicit address to map into lets you know that the address is just a suggestion, and it can't be guaranteed. This also goes for mmap() on POSIX.

I would suggest you write the program in such a way that a change in address doesn't matter. That is, don't store too many pointers to quantities inside the buffer, or if you do, patch them up after reallocation. Similar to the way you'd treat a buffer that you were going to pass into realloc().

Even the documentation for MapViewOfFileEx() explicitly suggests this:

While it is possible to specify an address that is safe now (not used by the operating system), there is no guarantee that the address will remain safe over time. Therefore, it is better to let the operating system choose the address. In this case, you would not store pointers in the memory mapped file, you would store offsets from the base of the file mapping so that the mapping can be used at any address.

Update from your comments

In that case, I suppose you could:

• Not map into contiguous blocks. Perhaps you could map in chunks and write some intermediate function to decide which to read from/write to?

• Try porting to 64 bit.

Answer 3

You could brute force it; suspend every thread in the process that isn't the one performing the mapping, Unmap/Remap, unsuspend the suspended threads. It ain't elegant, but it's the only way I can think of off-hand to provide the kind of mutual exclusion you need.