What's the fastest Perl IPC/message queue for a single machine?

Question

I'm working on a (primarily) Perl project and want to use a message queue to isolate processes from each other. I have a work flow like this:

Input -> Receiver -> Processor(s) -> Output(s)

I need to handle several hundred transactions/second, so speed my biggest motivator. What is the fastest message queue system for this type of setup?

All of my processes run on the same machine, so I can use UNIX sockets or temp-files if it means better performance.

So far my two favorites are IPC::Dirqueue and beanstalkd. Anyone have experience with both and can make conclusions about which is faster?

What else is out there that would make a super-fast work queue on a single machine?

Answer 1

I don't know about IPC::Dirqueue or beanstalkd, but I've had good success with spread (with either the Spread or Spread::Message modules). It's easy to do multi-process (with another daemon running) and also multi-machine. The nice thing is that since it's group based you can easily attach another listener if you want to do things like logging, statistics or debugging without having to alter (or slow down) your processors.