Can dirtiness of pages of a (non-shared) mmap be accessed from userspace under linux 2.6.30+? Platform-specific hacks and kludges welcome. Ideally, I'm looking for an array of bits, one per page (4kB?) of the mmap'ed region, which are set if that page has been written to since the region was mmap'ed. (I am aware, that the process doin...
I need the fastest way to periodically sync file with memory. What I think I would like is to have an mmap'd file, which is only sync'd to disk manually. I'm not sure how to prevent any automatic syncing from happening. The file cannot be modified except at the times I manually specify. The point is to have a checkpoint file which kee...
Mmap returns a void*, but not a volatile void*. If I'm using mmap to map shared memory, then another process could be writing to that memory, which means two subsequent reads from the same memory location can yield different values -- the exact situation volatile is meant for. So why doesn't it return a volatile void*? My best guess is ...
This simple python code: import mmap with file("o:/temp/mmap.test", "w+b") as fp: m = mmap.mmap(fp.fileno(), 0, access=mmap.ACCESS_READ|mmap.ACCESS_WRITE) m.write("Hello world!") Produces the following error (on the mmap.mmap(...) line): WindowsError: [Error 1006] The volume for a file has been externally altered so...
I've been having some trouble with FreeBSD and large mmaps. Linux does not show the same problems. On program startup it can always get the 1 GB map. However, there's a reload operation where the file is replaced and remapped. The new map is usually just a little bigger each time so it doesn't fit neatly into the old mmap location. This...
Hello, I have a UIImage and I would like to put its data in a file and and then used a mapped file to save some memory. Apparently, the UIImage data is private and it's not possible to access it. Would you have any suggestions to solve that? Thanks! ...
In our product we use a malloc implementation that relies exclusively on mmap for memory allocation. We also do a fair use of allocaing. We've just encountered a problem where mmap will allocate regions that should be reserved to stack space (thus causing very bad things to happen when one of our larger alloca's spills into the malloc'...
i have to make a driver in linux kernel which allocates buffers and provides them to userspace via mmap. How do i ensure that these buffers aren't cached? ...
I'm trying to find out how to remap memory-mapped files on a Mac (when I want to expand the available space). I see our friends in the Linux world have mremap but I can find no such function in the headers on my Mac. /Developer/SDKs/MacOSX10.6.sdk/usr/include/sys/mman.h has the following: mmap mprotect msync munlock munmap but no mre...
Is it possible to change the name of an already open memory mapped file, or, do I need to close it, rename it and then mmap it again? ...
I allocated some memory with anonymous mmap: buff->addr = mmap(NULL, length, PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS -1, 0); fprintf(stderr, "allocated buffer: %p, %lu\n", buff->addr, (unsigned long)length); then I'm writing to it using fd opened with O_DIRECT: int fd = open(name, O_CREAT | O_TRUNC | O_WRONLY | O_DIRECT, 00300); if(fd...
This happens on a 2.6.26-2-amd64 Linux kernel when trying to mmap a 5GB file with copy-on-write semantics ( PROT_READ | PROT_WRITE and MAP_PRIVATE). Mapping files smaller than 4GB or using only PROT_READ works fine. This is not a soft resource limit issue as reported in this question; the virtual limit size is unlimited. Here is the co...
I want to use a library that uses file descriptors as the basic means to access its data. For performance reasons, I don't want to have to commit files to the disk each before I use this library's functions. I want to create (large) data blobs on the fly, and call into the library to send them to a server. As it stands, I have to write ...
I'm running into an issue with GDB and some buffers allocated in kernel space. The buffers are allocated by a kernel module that is supposed to allocate contiguous blocks of memory, and then memory mapped into userspace via a mmap() call. GDB, however, can't seem to access these blocks at any time. For example, after hitting a breakpo...
Is there an option in boost memory maps that forces it to start bringing the entire file into memory without pagefaulting first? Is that operating system dependent or is will it be mostly portable? I need to bring entire files into memory and while I don't care about the actual contents I am concerned by the time it takes. I kind of don'...
I have some code (which I cannot change) that I need to get working in a native Win32 environment. This code calls mmap() and munmap(), so I have created those functions using CreateFileMapping(), MapViewOfFile(), etc., to accomplish the same thing. Initially this works fine, and the code is able to access files as expected. Unfortuna...
I have a program that uses Solaris' libumem memory allocator with mmap as a backend. That program does a lot of oversized (in terms of libumem slab concept) allocations. Does libumem in this situation cache oversized allocations, or are all those allocations made with fresh anon mmap() pages (and therefore initialized to zero)? ...
If I read and write a single file using normal IO APIs, writes are guaranteed to be atomic on a per-block basis. That is, if my write only modifies a single block, the operating system guarantees that either the whole block is written, or nothing at all. How do I achieve the same effect on a memory mapped file? Memory mapped files are ...
I'm wondering if there is a way to insert blank pages near the beginning of a large (multi-GB) file that I have open with mmap(). Obviously it would be possible to add a page or two to the end, and move everything forward with memcpy(), but this would dirty every page and require an awful long time when eventually flushed to disk. I'm ...
Hi there, I have a problem that was described in multiple threads concerning memory mapping and a growing memory consumption under Linux. When I open a 1GB file under Linux or MacOS X and map it into memory using me.data_begin = mmap(NULL, capacity(me), prot, MAP_SHARED, me.file.handle, 0); and sequentially read the mapped memory, m...