views:

456

answers:

4

It's widely known that the most significant mmap() feature is that file mapping is shared between many processes. But it's not less widely known that every process has its own address space.

The question is where are memmapped files (more specifically, its data) truly kept, and how processes can get access to this memory? I mean not *(pa+i) and other high-level stuff, but I mean the internals of the process.

+4  A: 

This happens at the virtual memory management layer in the operating system. When you memory map a file, the memory manager basically treats the file as if it were swap space for the process. As you access pages in your virtual memory address space, the memory mapper has to interpret them and map them to physical memory. When you cross a page boundary, this may cause a page fault, at which time the OS must map a chunk of disk space to a chunk of physical memory and resolve the memory mapping. With mmap, it simply does so from your file instead of its own swap space.

If you want lots of details of how this happens, you'll have to tell us which operating system you're using, as implementation details vary.

Charlie Martin
It'd be great if you could tell me what to read on this topic for UNIX. Exactly, it's Solaris OS.
Okay, cool. Then you want Mauro, McDougall and Greg's Solaris Internals. Check out the book wiki: http://www.solarisinternals.com/wiki/index.php/Solaris_InternalsBach's Design of the UNIX Operating System http://www.alibris.com/search/books/qwork/1604627/used/The%20Design%20of%20the%20UNIX%20Operating%20System and McKusick's book on BSD are good too. http://www.freebsd.org/doc/en/books/design-44bsd/
Charlie Martin
It's a great list. Thanks a lot for your assistance, Charlie.
NP, I've been teaching this crap forever.
Charlie Martin
A: 

I'm not really sure what you are asking, but mmap() sets aside a chunk of virtual memory to hold the given amount of data (usually. It can be file-backed sometimes).

A process is an OS entity, and it gains access to memory mapped areas through the OS-proscribed method: calling mmap().

T.E.D.
A: 

The kernel has internal buffers representing chunks of memory. Any given process is assigned a memory mapping in its own address space which refers to that buffer. A number of proccesses may have their own mappings, but they all end up resolving to the same chunk (via the kernel buffer).

This is a simple enough concept, but it can get a little tricky when processes write. To keep things simple in the read-only case there's usually a copy-on-write functionality that's only used as needed.

dwc
+1  A: 

This is very implementation-dependent, but the following is one possible implementation:

When a file is a first memory-mapped, the data isn't stored anywhere at first, it's still on disk. The virtual memory manager (VMM) allocates a range of virtual memory addresses to the process for the file, but those addresses aren't immediately added to the page table.

When the program first tries to read or write to one of those addresses, a page fault occurs. The OS catches the page fault, figures out that that address corresponds to a memory-mapped file, and reads the appropriate disk sector into an internal kernel buffer. Then, it maps the kernel buffer into the process's address space, and restarts the user instruction that caused the page fault. If the faulting instruction was a read, we're all done for now. If it was a write, the data is written to memory, and the page is marked as dirty. Subsequent reads or writes to data within the same page do not require reading/writing to/from disk, since the data is in memory.

When the file is flushed or closed, any pages which have been marked dirty are written back to disk.

Using memory-mapped files is advantageous for programs which read or write disk sectors in a very haphazard manner. You only read disk sectors which are actually used, instead of reading the entire file.

Adam Rosenfield
Thank you. Very useful addition about page fault mechanism, I've totally forgotten about this technique.