the operation should be: - one queue, many consumers are ready, but only one active at time; - if a consumer fails the other should be elected; - each consumer in a different host; - must be sure that only one consumer will act at a time; What is the best strategy to ensure the uniqueness in the operation and also the contingency of ope...
Does .NET 4 Parallel Extensions include an implementation of a lock-free Producer/Consumer queue? Is there any class or interface to implement which could help me to implement this pattern correctly? Thanks! ...
I have to work with some legacy code written in a "push" callback style. The architecture looks like this (pseudo-Java) public interface Callback { public void consumeData(Object data); } public class Worker // The legacy class { private Callback cb; public Worker(..., Callback cb) { this.cb = cb; ... ...
I've written background InputStream (and OutputStream) implementations that wrap other streams, and read ahead on a background thread, primarily allowing for decompression/compression to happen in different threads from the processing of the decompressed stream. It's a fairly standard producer/consumer model. This seems like an easy wa...
Hello All I have a problem with messaging (with MSMQ) which is a variation of fast producer/slow consumer. Is there a way to get outstanding unconsumed message count in a private MSMQ queue? I would like to use that to throttle the producer. I would like to use a semaphore paradigm with MSMQ where the producer application will only s...
I'm wrestling with the best way to implement my processing pipeline. My producers feed work to a BlockingQueue. On the consumer side, I poll the queue, wrap what I get in a Runnable task, and submit it to an ExecutorService. while (!isStopping()) { String work = workQueue.poll(1000L, TimeUnit.MILLISECONDS); if (work == null) ...
I have two processes: Writes to two tables every second (ish) Reads from said tables periodically I know that with SQLite, any writes lock the whole database and so sometimes the second process can fail with a locked database. Is there anything you can suggest that would completely remove the need for these two processes to touch th...
Hello, I'm looking for a way to use pthread rwlock structure with conditions routines in C++. I have two questions: First: How is it possible and if we can't, why ? Second: Why current POSIX pthread have not implemented this behaviour ? To understand my purpose, I explain what will be my use: I've a producer-consumer model dealing ...
I'm preparing for an interview in a few weeks and I thougth I would give threads in boost a go, as well as do the simple producer/consumer problem I learned in school. Haven't done it quite awhile so I was curious what you guys think of this? What should I add to make it a better example etc. Thanks for the feedback! :) ///////////////...
Hi all, I'm trying to implement a Consumer/Producer app in Grails, after several unsuccessful attempts at implementing concurrent threads. Basically I want to store all the events coming from a clients (through separate AJAX calls) in a single queue and then process such a queue in a linear way as soon as new events are added. This look...
I'm starting with the C# code example here. I'm trying to adapt it for a couple reasons: 1) in my scenario, all tasks will be put in the queue up-front before consumers will start, and 2) I wanted to abstract the worker into a separate class instead of having raw Thread members within the WorkerQueue class. My queue doesn't seem to disp...
With alle the new paralell programming features in .NET 4.0, what would be a a simple and fast way to implement the producer-consumer pattern (where at least one thread is producing/enqueuing task items and another thread executes/dequeues these tasks). Can we benfit from all these new APIs? What is your preferred implementation of this ...
Hi all, I have five consumers and one producer. The five consumers each output different data, from the one producer, for ~10ms. During those 10ms the producer prepares the parameters for the next output. When the output parameters are set, I want to set a flag instructing the consumers to begin the next output. I only want the producer...
I have a producer-consumer like scenario. Class A produces objects of type E. I have to hold it in a static data structure in class A, because the consumer logic should be handled in a class B, which has no reference to object of A. Is LinkedBlockingQueue the correct data type for the queue? Or are there any better selection for this? ...
I have a question about Producer/consumer Design pattern , in fact my situation is :I have One class that produce multiple types of messages (notifications) and multiple consumers who consume those messages. The complication is my Producer produce different types of messages and my consumers consume those messages . So what is the best...
Hi Folks, this is some kind of long post, so I have to say thanks for reading. My app is supposed to process a lot of soundfiles, lets say 4000+. My first approach was to load a certain amount (lets say 200mb) of sound data, process it, write it and then "null" the data to let the gc free it. But regarting to the fact that the data is ...
I initially asked this question here, but I've realized that my question is not about a while-true loop. What I want to know is, what's the proper way to do high-performance asynchronous message-passing in Java? What I'm trying to do... I have ~10,000 consumers, each consuming messages from their private queues. I have one thread that'...
while (true) { BasicDeliverEventArgs e = (BasicDeliverEventArgs)Consumer.Queue.Dequeue(); IBasicProperties properties = e.BasicProperties; byte[] body = e.Body; Console.WriteLine("Recieved Message : " + Encoding.UTF8.GetString(body)); ch.BasicAck(e.DeliveryTag, ...
I'm in a situation where i have to ship a product, a php-based product. Sorry for the general information cause what i really need is an general approach, an approach that will work for any php-product. ...
I am trying to make a producer/consumer thread situation more efficient by skipping expensive event operations if necessary with something like: //cas(variable, compare, set) is atomic compare and swap //queue is already lock free running = false // dd item to queue – producer thread(s) if(cas(running, false, true)) { // We effect...