What is the best to format messages for queueing?

Question

I've been reading up on message queueing lately, and I'd like to implement a simple, extendable, system for my app. While there's a lot of good information on the subject of setting up a MQ system out there, I can't find a lot about the actual implementation.

I'm looking for patterns and best practices on how to properly format messages for a queue, and ways to execute the jobs in PHP. Should I use JSON, serialized objects, text, URLs or XML? What information should I send? Is a worker with a switch($job['command']) {} (or something like that) the way to go, or are there any established patterns out there to implement a worker?

Help greatly appreciated!

Answer 1

You can pick any of the following MQ implementations in PHP, so you don't have to roll your own and you can look at their sourcecode to learn about their implementation. For general integration, have a look at the ActiveMQ page on Enterprise Integration patterns.

• http://sourceforge.net/projects/beanstalk/

  A PHP Client Library for beanstalkd. BeanStalk allows PHP developers to make use of the beanstalkd in-memory workqueue server (http://xph.us/software/beanstalkd).

• http://kr.github.com/beanstalkd/

  Beanstalk is a simple, fast workqueue service. Its interface is generic, but was originally designed for reducing the latency of page views in high-volume web applications by running time-consuming tasks asynchronously.

• http://activemq.apache.org/

  Apache ActiveMQ is the most popular and powerful open source messaging and Integration Patterns provider. Apache ActiveMQ is fast, supports many Cross Language Clients and Protocols, comes with easy to use Enterprise Integration Patterns and many advanced features while fully supporting JMS 1.1 and J2EE 1.4. Apache ActiveMQ is released under the Apache 2.0 License

• http://memcachedb.org/memcacheq/

  Memcachedb is a distributed key-value storage system designed for persistent. It is not a cache solution, but a persistent storage for high-frequency writing and reading. It conforms to memcache protocol(not completed, see below), so any memcached client can have connectivity with it. Memcachedb uses Berkeley DB as a storing backend, so lots of features including transaction and replication are supported.

• http://www.zend.com/en/products/server/

  Zend Server 5.0 incorporates Job Queue, providing full support for creating, executing and managing jobs to optimize application performance and reduce server load, minimizing application bottlenecks and improving the end-user experience.

• https://www.dropr.org/

  dropr is a distributed message queue framework written in PHP. The main goals are:

  • reliable and durable (failsafe)-messaging over networks
  • decentralized architecture without a single (point of failure) server instance
  • easy to setup and use
  • modularity for queue storage and message transports (currently filesystem storage and curl-upload are implemented)

• http://gearman.org/

  Gearman provides a generic application framework to farm out work to other machines or processes that are better suited to do the work. It allows you to do work in parallel, to load balance processing, and to call functions between languages.

Answer 2

I would use JSON if you can--it's concise, easy to read, and works cross-language.

One reason you might not be able to use JSON is if you want your background "job" to include not just data (in which case JSON will be fine), but a serialized object. If this is the case, you'll need to make at least one of the job's arguments serialized PHP.