message passing and signaling in distributed system

Question

I have a distributed video analysis system, which is composed of:

1. feature extraction: generated lots of features(20+) from each frame of the video
2. multiple detectors(in different machine): 
  * Each of them will get a subset of feature
  * Each of them needs the features from multiple frames. 
  * Eg. Detector 1 needs feature 1-5 from 3 frames to start processing; Detector 2 needs feature 2-8 from 8 frames to start processing

My question is: how to do the communication between feature extraction block and multiple detectors, preferably in real-time? I have been looking at event bus, but it's only for one process, will ZooKeeper in Hadoop be a better solution?

I am using Java. Any suggestion is welcomed.

Answer 1

In Java, you could consider using RabbitMQ message bus. There is a library with Java bindings.

RabbitMQ is based on a proven platform, offering exceptionally high reliability, availability and scalability along with good throughput and latency performance that is predictable and consistent. It has a compact, easily maintainable code base allowing rapid customisation and hot deployment. There are extensive facilities for management, monitoring, control and debugging and it is supported by a full range of commercial support services and an active community developing packages that extend the core system.