Recommended Python publish/subscribe/dispatch module ?

Question

From PyPubSub:

Pypubsub provides a simple way for your Python application to decouple its components: parts of your application can publish messages (with or without data) and other parts can subscribe/receive them. This allows message "senders" and message "listeners" to be unaware of each other:

• one doesn't need to import the other
• a sender doesn't need to know
  
  • "who" gets the messages,
  • what the listeners will do with the data,
  • or even if any listener will get the message data.
• similarly, listeners don't need to worry about where messages come from.

This is a great tool for implementing a Model-View-Controller architecture or any similar architecture that promotes decoupling of its components.

There seem to be quite a few Python modules for publishing/subscribing floating around the web, from PyPubSub, to PyDispatcher to simple "home-cooked" classes.

Can you recommend a module that works well in most cases ? Which modules have you had positive experience with ? Negative ?

Thanks in advance

Answer 1

PyDispatcher is used heavily in Django and it's working perfectly for me (and for whole Django community, I guess).

As I remember, there are some performance issues:

• Arguments checking made by PyDispatcher is slow.
• Unused connections have unnecessary overhead.

AFAIK it's very unlikely you will run into this issues in a small-to-medium sized application. So these issues may not concern you. If you think you need every pound of performance (premature optimization is the root of all evil!), you can look at modifications done to PyDispatcher in Django.

Hope this helps.

Answer 2

I recently looked carefully at py-amqplib to act as an AMQP client to a RabbitMQ broker. The latter tool is written in Erlang.

If you're looking to decouple your app. then why couple it to the language itself? Consider using message queues which are language neutral and then you've really got room to grow!

That being said, AMQP takes effort to understand and may be more than you are willing to take on if your app. is working just fine as is. YMMV.

Answer 3

There is also the libraries by PJ Eby, RuleDispatch and the PEAK project, specially Trellis. I don't know what their status actually but the mailing list is quite active.

Last version of Trellis on PyPi

Trellis doc

I have also used the components from the Kamaelia project of the BBC. Axon is an interesting approach, but more component than publisher-consumer inspired. Well, its website is somewhat not up-to-date at all... There was a project or 2 in the Google SoC 2008 and work is being done.

Don't know if it help :)

Edit : I just found Py-notify which is an "unorthodox" implementation of the Observer pattern. It has most of the functionalities that I need for my own tools.

Answer 4

The fact alone that PyPubSub seems to be a somewhat chaotically managed project (the Wiki on SF is dead, the website (another Wiki) which is linked on SF is currently broken) would be enough reason for me not to use it. PyDispatcher has an intact website, but the only documentation they seem to provide is the one for the API generated from the docstrings. No traffic on the mailing list either... a bad sign!

As Mike also mentioned, it's perfectly possible to choose a solution that is independent of Python. Now don't get me wrong, I love Python, but still, in this field it can make sense use a framework that is decoupled from the programming language.

I'm not experienced with messaging, but I'm planning to have a look into a few solutions. So far these two (free, open source) projects seem to be the most promising for me (coincidentally, both are Apache projects):

• ActiveMQ
• Qpid

Both seem to be reasonably matured projects, at least a far as documentation and community. I can't comment on the software's quality though, as I said, I didn't use any of the software.

Qpid ships with client libraries for Python, but you could also use py-amqplib. For ActiveMQ there's pyactivemq, which you can use to connect either via STOMP (Streaming Text Orientated Messaging Protocol) or via Openwire.

Answer 5

Some libraries I have found that haven't yet been mentioned:

• Circuits - a Lightweight, Event driven Framework with a strong Component Architecture.
• C# Event Recipe

Answer 6

The best dispatch package for python seems to be the dispatch module inside django (called signals in the documentation). It is independent of the rest of django, and is short, documented, tested and very well written.

(for the record, there is also Louie, which is one of the descendant of PyDispatcher)