views:

413

answers:

6

I am trying to create a data buffer, more specifically, an image buffer, which will be shared among multiple modules. Those modules only reads from the buffer and don't communicate with each other at all. My difficulty is:

1.Large data size:

larger than 10M per image, that means copying those data around for different threads is not desirable

2.I don't want the memory grow wild:

as new data continuously come in(in real-time), the very old data must be deleted when all the modules have finished using it.

However, to make things even more complex, those modules who consumes the data are at different pace: some are faster/slower, some needs more data(multiple images) to get a result, some need less(only one image)

I have been thinking about using shared_ptr to solve the 1st problem: create a queue of boost shared_ptr(s), each shared_ptr() points to an image(char array). Then pass a subset of those pointers to different module.

I am a totally newbie in this smart pointer field. What will be a good solution for this problem?

Thanks.

+3  A: 

A Boost shared pointer is exactly what I was going to suggest. Yes, let the pointer class do the work for you.

Note that you will want to use boost::shared_array instead of shared_ptr if you are storing array pointers.

The shared_array class template stores a pointer to a dynamically allocated array. (Dynamically allocated array are allocated with the C++ new[] expression.) The object pointed to is guaranteed to be deleted when the last shared_array pointing to it is destroyed or reset.

John Kugelman
ya, you are right. for char array, shared_array is the one I need.
Lily
+1. Shared ptr is the way to do this.
Soo Wei Tan
+2  A: 

Assuming you hand the shared_ptrs to the modules as soon as the buffer is created, they are a good fit. You don't even need to store them centrally in that case.

It gets more complicated however, if you create the buffers at one point and only at some other point later the modules request the buffer.
In that case you have to figure out what behaviour you want.
Do you want to hold the buffers for some time? Or until at least one module has used them? Or until some new data comes in?

integration of comment:
As you want all your readers/modules to handle all incoming data you can simply give these an input queue. On incoming data just hand the modules an shared_ptr/shared_array to the new buffer, which add them to the queue.
Remember to handle the multi-threading issues though for the queue access.

Georg Fritzsche
In my case, I can't assume the speeds of writer(data feed)/reader(reading+processing modules). Best scenario is that when new data comes in, readers are all ready to read. However, if the reader is slower, either I need to buffer them centrally or the separate modules needs to buffer them separately. So the life cycle will be: if needed, hold the buffer for some time until all modules have done with it.
Lily
It would be the simplest to just have an input queue in every reader which holds the incoming `shared_ptr`s/`shared_array`s.
Georg Fritzsche
65you mean: 1) get rid of central queue, but replace it with one queue per reader. 2) I am not quite sure about the multi-threading issue you mentioned. The only issue I could think about is that I shouldn't read/writer at the same time. Cauz' all the other thing are handled by shared_ptr already. Am I correct?
Lily
1) yes. 2) shared_ptrs and shared_arrays are not completely thread-safe, see: http://www.boost.org/doc/libs/1_40_0/libs/smart_ptr/shared_ptr.htm#ThreadSafety
Georg Fritzsche
To clarify: I will have one data feed(1 thread), it generates 4 shared_ptr(s) and signal 4 writers, pushing the new shared_ptr into 4 queues respectively. The each queue has new item and the amount of data is enough(some processing module need 3 shared_ptr to start processing), it signal its worker to process the data. Hopefully I fully understand your suggestion. ;-)
Lily
That sounds about right.
Georg Fritzsche
A: 

1.Large data size:

You are correct in choosing to store the image data in heap allocated buffers, and then passing pointers to them between your processing modules.

2.I don't want the memory grow wild

You don't have to use a queue for memory management if you are using shared_ptr(). Design your modules to create/accept a shared_ptr() when it needs access to the data, and when it is done, to delete the shared_ptr(). The intention of a shared_ptr() is that the heap memory owned by the pointer is deleted when there are no more references to it.

Phillip Ngan
I don't quite get you on your comment for 2. If I don't use a queue(or other container class) to hold those pointers, how can those modules accept them, esp when some of the modules need more than one pointer... Did I think it wrong?
Lily
I guess he assumes handing the shared_ptrs immediately going to be used in all modules, thus you have the shared_ptrs already managing the memory.
Georg Fritzsche
A: 

save the image to a file so you could try posix file mapping to map to memory per image. after mapping, you could make it as the shared memory to be used efficiently even among multi-processes.

Btw: does your system support posix file mapping? e.g. mmap in Linux etc.

EffoStaff Effo
+1  A: 

According to your requirements, I think you could use two principles:

  • shared_array<char> which will handle the multi-thread synchronization and memory-handling
  • one queue per module: this one is necessary since each module is dealing with the images at its own pace

Then, as soon as you get an image, you allocate it on the heap in a shared_array<char>. This pointer is then replicated in all the queues.

Each queue individually requires synchronization, it's a classic Consumer / Producer thing though, so you'll probably program it (quite) easily, especially since each queue only have ONE producer (the thread which receives the image) and ONE consumer.

Let's have an example: let's take 3 modules, one is fast, one is medium and the last use the images 3 by 3.

=> receiving image 1
module a: ['1'] -> processing (none)
module b: ['1'] -> processing (none)
module c: ['1'] -> processing (none)

=> modules a, b starts treatment of '1'
module a: [] -> processing '1'
module b: [] -> processing '1'
module c: ['1'] -> processing (none)

=> receiving image 2
module a: ['2'] -> processing '1'
module b: ['2'] -> processing '1'
module c: ['2', '1'] -> processing (none)

=> module a finishes treatment of '1', starts treatment of '2'
module a: [] -> processing '2'
module b: ['2'] -> processing '1'
module c: ['2', '1'] -> processing (none)

=> receiving image 3
module a: ['3'] -> processing '2'
module b: ['3', '2'] -> processing '1'
module c: ['3', '2', '1'] -> processing (none)

=> module c starts treatment of '1', '2' and '3'
module a: ['3'] -> processing '2'
module b: ['3', '2'] -> processing '1'
module c: [] -> processing '1', '2' and '3'

=> module a finishes treatment of '2', starts treatment of '3'
=> module b finishes treatment of '1', starts treatment of '2'
=> module c finishes treatment of '1' and '2', keeps '3' for future batch
module a: [] -> processing '3'
module b: ['3'] -> processing '2'
module c: [] -> processing '3' (waiting)

--> at this point '1' is deleted from memory

You can even make this 'easy' if each module (thread) registers its queue in a 'pool'.

I would also advise signalling, I always think it was better for a producer to signal that a new item had been inserted (if the queue was empty) that having the consumer thread constantly polling the queue...

Matthieu M.
@queue pool: Following your suggestion, I should create a thread pool for each of the modules. There can be multiple workers working on one module's work. I have never used it before, so would you mind clarify? I found a [threadpool class](http://threadpool.sourceforge.net/design/pattern.html). I am also looking at: [Boost Pool class](http://www.boost.org/doc/libs/1_37_0/libs/pool/doc/interfaces.html)
Lily
@signalling, I am totally agree. It's more like a classic producer/customer pattern, and the producer should notify all the customers that new item added. What I really don't know is how to combine this signaling and shared_ptr generation with this threadpool idea.
Lily
@queue pool: I was not talking about thread pool, a single thread can hold the multiple queues (if they are synchronized), the idea is to have one queue per module since you said that each module consumed the items at its own rythm.
Matthieu M.
@signalling: the idea would be to first generate the shared_array when receiving the data, then put it in each queue one after the other and sending a signal to the module to which the queue corresponds if the queue was empty before the insertion so as to 'wake' it up.
Matthieu M.
@queue pool: also, bear in mind that even if the 'main' module thread pops the shared_array from the queue, and then delete its own copy of the shared_array, the workers can still work on it independently as long as they were given a copy. That's the beauty of shared_array.
Matthieu M.
that means I have one data feed, say I have 4 modules/readers, then I also need 4 writers to write to different queues, to enable the signalling.
Lily
Yes that's the idea, if you are afraid of performance problem (say you may receive 'burst' of data), then you could have another queue to act as temporary buffer: ie your receiver queues the item for the duplicator, which in turn duplicate this item accross the 4 modules and does the signalling appropriately. But if you can do without, that would be simpler.
Matthieu M.
A: 

Use boost::shared_array as data container (John suggestion). And boost::circular_buffer as input queue in your modules.

boost::circular_buffer< const boost::shared_array<char> > input_queue_;

As images are shared you should not modify them but make a new copy when needed.

fnieto
I was thinking about circular_buffer before, too(when I tried to implement the central queue). However, I am not quite sure about central buffer vs separate buffers for each processing module.
Lily
As long as modules consume data are at different rate, you will need a buffer for each input in a module. You don't need to worry too much about memory consumption because it is limited by the circular buffers size (you have control) and the fact of having just references (not copies) of each image.
fnieto