I have a very large array of doubles that I am using a disk-based file and a paging List of MappedByteBuffers to handle, see this question for more background. I am running on Windows XP using Java 1.5. Here is the key part of my code that does the allocation of the buffers against the file... try { // create a random access file an...
Hi all, I'm trying to memory-map and use a largish binary data file from my iPhone bundle resources. I've added the "Resources should be file-mapped" line with a checkmark to my Info.plist. Then I'm getting the data block like this (error handling and data release omitted): CFBundleRef bun = CFBundleGetMainBundle(); CFURLRef url = CFBu...
I am trying to understand how Boost memory mapped files work. The following code works, it does what it is supposed to do, but the problem is that the file it generates is stored on disk (in the same directory of the executable) instead of memory. Maybe there is a flag to set somewhere, but I could not find it... Thanks in advance for an...
Hi, I'm playing with C dlls to hook global Windows events and the next step is sending some event data (nothing huge) to a C# application. As I want this communication to be as fast as possible, I'm analysing two options: Named Pipes and Memory Mapped Files. I know that .NET 4 brings MMF in a native way, but I have to target .NET 2, a...
I have been looking into using Boost.Interprocess to create a disk-backed data structure. The examples on Boost Documentation (http://www.boost.org/doc/libs/1_41_0/doc/html/interprocess.html) are all for using shared memory even though they mention that memory mapped files can also be used. I am wondering whether anyone here has used mem...
I want to pose a seemingly simple question that i can't find the answer nowhere. Is there a FAST modern algorithm for file input and/or output that can be compiled with all standard compliant C++ compilers and works for all operating systems without the requirement of external libraries ? i have found that the fastest way is to use mem...
Considering a really huge file(maybe more than 4GB) on disk,I want to scan through this file and calculate the times of a specific binary pattern occurs. My thought is: Use memory-mapped file(CreateFileMap or boost mapped_file) to load the file to the virtual memory. For each 100MB mapped-memory,create one thread to scan and calculate...
The following C code illustrates a problem I'm seeing on Linux 2.6.30.5-43.fc11.x86_64: #include <sys/types.h> #include <sys/stat.h> #include <sys/mman.h> #include <fcntl.h> #include <unistd.h> #include <stdlib.h> #include <stdio.h> #include <string.h> int main() { char buf[1024]; void *base; int fd; size_t pagesz = sys...
Hi. Having learnt a bit about the subject, can anyone tell, what is the real difference between POSIX shared memory (shm_open) and POSIX mapped files (mmap)? Both seems to use the /dev/tmpfs subsystem, rather then older IPC mechanism. So is there any advantage of using mmap file over shared memory? Thanks. ...
I'd like to know if anyone tried new .NET 4.0 Memory Mapped Files features? I know that they are as old as OS but native handling in .NET is new. Has anyone been able to test this and say something about performance? I'm quite interested in access time to random parts of binary file, write speed and so on. Also performance comparsion ...
Microsoft introduced a new System.IO.MemoryMappedFiles namespace in .Net 4. Does anyone know if it will support alternate data streams? Thx! ...
Hi, I'm trying to use MapViewOfFile in a 64 bit process on a file that is already mapped to memory of another 32 bit process. It fails and gives me an "access denied" error. Is this a known Windows limitation or am I doing something wrong? Same code works fine with 2 32bit processes. The code sort of looks like this: hMapFile = Open...
I have two applications (processes) running under Windows XP that share data via a memory mapped file. Despite all my efforts to eliminate per iteration memory allocations, I still get about 10 soft page faults per data transfer. I've tried every flag there is in CreateFileMapping() and CreateFileView() and it still happens. I'm begin...
Or it would be faster to re-read that data from mapped memory once again, since the OS might implement its own cache? The nature of data is not known in advance, it is assumed that file reads are random. ...
Scenario I have the following methods: public void AddItemSecurity(int itemId, int[] userIds) public int[] GetValidItemIds(int userId) Initially I'm thinking storage on the form: itemId -> userId, userId, userId and userId -> itemId, itemId, itemId AddItemSecurity is based on how I get data from a third party API, GetValidItemI...
I'm trying to illustrate the difference in performance between traditional IO and memory mapped files in java to students. I found an example somewhere on internet but not everything is clear to me, I don't even think all steps are nececery. I read a lot about it here and there but I'm not convinced about a correct implementation of neit...
I need to store a dictionary to a file as fast as possible. Both key and value are objects and not guaranteed to be marked as Serializable. Also I prefer a method faster than serializing thousands of objects. So I looked into Mapped Memory Files support in .NET 4. However, it seems MemoryMappedViewAccessor only allows storage of structs ...
Greeting All, I want to achieve transparent persistence of Java objects through memory-mapped files (utilize the OS paging/swapping mechanism). My problem is: how can I move a Java object to my memory-mapped block ? Plus, how can I force a new object instance to reside in such blocks ? As you all know, a memory-mapped block can be see...
I am mapping a file("sample.txt") to memory using FileChannel.map() and then closing the channel using fc.close(). After this when I write to the file using FileOutputStream, I am getting the following error: java.io.FileNotFoundException: sample.txt (The requested operation cannot be per formed on a file with a user-mapped sec...
Does this function affect Memory-mapped file performance? Here's the problem I need to solve: I have two applications competing for disk access: "reader" and "updater". Whole system runs on Windows Server 2008 R2 x64 "Updater" constantly accesses disk in a linear manner, updating data. They system is set up in such a way that updater ...