mmaping two consecutive pages

Question

I'm writing a unit test for my UTF8 manipulation library, and I want my test to segfault if a function goes into a buffer overflow. So I came up with the idea to mmap two pages next to each other in memory, the first with PROT_READ | PROT_WRITE, and the second with PROT_NONE. That way, if any overflow occurs, a segfault is guaranteed. Here's an example:

void *addr1, *addr2; /* these are the pages; mmap call left out for simplicity */
char *p = (char *) (addr1 + getpagesize() - 8);

utf8_encode(aUtf8String, p, 8); // this shouldn't segfault

The problem is, when I map the second page, my program segfaults. Here's an example program that reproduces the problem (GNU/Linux):

#include <stdio.h>
#include <errno.h>
#include <string.h>
#include <stdlib.h>
#include <sys/mman.h>

void checkMap(void *p) 
{ 
    if(p == MAP_FAILED) {
        printf("error running mmap: %s\n", strerror(errno));
        exit(1);
    }   
}

int main(void)
{
    void *addr1, *addr2;
    size_t pagesize;

    pagesize = getpagesize();
    checkMap(addr1 = mmap(NULL, pagesize, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0));
    checkMap(addr2 = mmap(addr1 + pagesize, pagesize, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE | MAP_FIXED, -1, 0));  /* segfaults */
    munmap(addr1, pagesize);
    munmap(addr2, pagesize);

    return 0;
}

Interestingly enough, a printf() statement before the first mmap() causes the program to run successfully. Does anyone know why mmap is segfaulting? If my goal is unattainable using mmap(), does anyone have any other advice on how I could test my code for buffer overflow?

Answer 1

You can call mprotect() to change the protection flags on memory mapped by mmap(). This might be a better solution than trying to mmap() two adjacent pages with different protections, since this seems to be what's causing your problems.

(Linux allows you to call mprotect() on any page, but POSIX only allows pages that have already been allocated by mmap().)

This is one of the tricks Electric Fence uses to catch buffer overruns.

Answer 2

This is not completely related to your question (unless your ultimate goal is indeed to get your test working), but, IMHO, relying on this kind of fault handling might be misleading (most notably if you're targeting multiple platforms, where the test should've failed just that the seg-fault wasn't triggered due to some unspecified behavior).

Perhaps a different approach would make more sense for you, for example, simply allocate a larger buffer than necessary, place a marker at the end of it, and check if it has been overwritten?

As others have mentioned, if you're willing to set up a more complicated testing environment, electric fence, valgrind, or other tools are probably more elaborate in their analysis.