Boost.MultiIndex: Are there way to share object between two processes?

Question

Hello, I have a Boost.MultiIndex big array about 10Gb. In order to reduce the reading I thought there should be a way to keep the data in the memory and another client programs will be able to read and analyse it.

What is the proper way to organize it?

The array looks like:

    struct particleID
    {
    int           ID;// real ID for particle from Gadget2 file "ID" block
    unsigned int  IDf;// postition in the file
    particleID(int id,const unsigned int idf):ID(id),IDf(idf){}
    bool operator<(const particleID& p)const { return ID<p.ID;}
    unsigned int getByGID()const {return (ID&0x0FFF);};

    };

struct ID{};
struct IDf{};
struct IDg{};

typedef multi_index_container<
    particleID,
    indexed_by<
        ordered_unique<
            tag<IDf>,  BOOST_MULTI_INDEX_MEMBER(particleID,unsigned int,IDf)>,
        ordered_non_unique<
            tag<ID>,BOOST_MULTI_INDEX_MEMBER(particleID,int,ID)>,
        ordered_non_unique<
            tag<IDg>,BOOST_MULTI_INDEX_CONST_MEM_FUN(particleID,unsigned int,getByGID)> 
    >
> particlesID_set;

Any ideas are welcome.

kind regards Arman.

EDIT: The RAM and the number of cores are not limited. Currently I have a 16Gb and 8cores.

Update

The same question I was asking in Boost.Users forum I got an answer from Joaquín M López Muñoz(developer of Boost.MultiIndex). The aswer is Yes. One can share the multi_index between processes using Boost.Interprocess. For more detail you can see in this link

Answer 1

Have you looked at Boost.Interprocess?

Answer 2

Have you thought about cutting it into pieces.

Concurrent access is hard. Hard to get right, hard to maintain, hard to reason about.

On the other hand, 10GB is very big, and I wondered if you could cluster your data. Keep the same index structure but dispatch it into 10 (or more) independent objects depending on some conditions (the big id for example).

This way you could naturally process each chunk separately from the other without having to deal with concurrent access in the first place.