Memory Based Data Server for local IPC

Question

I am going to be running an app(s) that require about 200MB of market data each time it runs. This is trivial amount of data to store in memory these days, so for speed thats what i want to do.

Over the course of a days session I will probably run, re-run, re-write and re-run etc etc one or more applications over and over.

SO, the question is how to hold the data in memory all day such that even if the app crashes I do not have to reload the data by opening the data file on disk and re-loading the data?

My initial idea is to write a data server app that does nothing more than read the data into shared memory so that it is available for use. If I do that I guess I could use memory mapping for the IPC by calling

CreateFile()

CreateFileMapping()

MapViewOfFile()

Is there a better IPC/approach?

Answer 1

Just memory map the data file. Unless your computer is low on memory, the file will stay in file cache even when the program exits. The next time it starts up, access will be fast.

If your in-memory data is different from the on-disk data, just use two files. On restart, check a timestamp and a file revision written into the memory file to compare to the disk file and that way your program will know which one has the most recent data.

Answer 2

If you have enough memory and nothing else asks for memory, that might reduce your startup time. To guarantee access to the memory, you probably want to have a memory mapped file in named shared memory, as described here. You can have a simple program create the share and manage it so you can guarantee it remains in memory.