32-bit FreeBSD 7.2 and 1GB mmaps

Question

I've been having some trouble with FreeBSD and large mmaps. Linux does not show the same problems.

On program startup it can always get the 1 GB map. However, there's a reload operation where the file is replaced and remapped. The new map is usually just a little bigger each time so it doesn't fit neatly into the old mmap location. This remap often fails on FreeBSD but hardly ever on Linux. (It fails more often on Linuxes using ASLR. Does FreeBSD use ASLR?)

I think this is because of libraries loading other libraries and also memory allocations done during the program run that fragment the virtual memory space. There is probably some peculiarity of FreeBSD's malloc implementation or library loading that causes this.

I'm looking for some things I can do to make it more likely to work.

One thing I am considering is to just always mmap a full 1 GB of space even if the file size might actually be 890 MB. If I can make that work then I will always have a 1 GB slot to remap the new 895 MB file into.

Answer 1

The problem is probably caused by not having a big enough contigous block of address space. It's very likely on a 32-bit system with a 1Gb mmap.

Consider using 64-bit systems exclusively (most companies do for production servers now), or mmap()ing several smaller areas instead.

You aren't going to get around this.