views:

438

answers:

3

I have a website written in PHP (CakePHP) where certain resource intensive tasks are handled by a background process. This is done through the Beanstalkd message queue. I need some way to retrieve the status of that background process so I can monitor it with Monit.

The background process is a CakePHP Shell (just a PHP CLI script) that communicates with Beanstalkd. It simply does a reserve() on Benastalkd and waits for a new message. When it gets a message, it processes it. I want some way of monitoring this process with Monit so that it can restart the background process if something has gone wrong.

What I have been thinking about so far is writing a PHP CLI script that drops a message in Beanstalkd. The background process picks up the message and somehow communicates it's internal status back to the CLI script. But how? Sockets? Shared memory? Some other IPC method?

Or am I perhaps being too complicated here and is there a much easier way to monitor such a process with Monit?

Thanks in advance!

A: 

There probably is a plugin to Monit or Nagios to connect, run the stats and return if there are 'too many'. There isn't a 'protocol' written already for that, but t doesn't appear to be exceeding difficult to modify an existing text-based one (like nntp, or smtp) to do what you want. It does mean writing it in C though, by the looks of it.

From a CLI-PHP script, I would go about it through one (or both) of two different methods.

1/ drop a (low-ish) priority message into the queue, and make sure it comes back within a few seconds. Putting it into a dedicated queue and making sure there's nothing there before you put it in there would be a good addition as well. 2/ perform a 'stats' and see how many are waiting: 'current-jobs-ready'.

To get the information back to a website (either way), you can write to a file, or into something like Memcached which gts read and acted upon.

Alister Bulman
Not quite what I am looking for. Your suggestion seems to test beanstalkd, not my background process that is consuming beanstalkd messages. So, I somehow need to signal my background process (either through beanstalkd, or using POSIX signals. I don't know any other way) and then the background process needs to communicate it's status back. But how?
Sander Marechal
If you want to make sure the worker keeps running I would tend to have the background process put a note into a cache (eg, memcached) as to results it produces and its own status. It could put an short-life message into memcached with a time it last ran for example. To make sure it keeps running, I wrap the worker in something like the bash script at http://www.topbit.co.uk/serendipity/archives/22-Doing-the-work-elsewhere-Sidebar-running-the-worker.html
Alister Bulman
+1  A: 

Here's what I ended up doing in the end.

The CLI script connects to beanstalkd, creates a new queue (tube) and starts watching it. Then it drops a highest priority message in the queue that the background daemon is watching. That message contains the name of the new queue that the CLI script is monitoring.

The background process receives this message almost immediately (because it is highest priority), generates a status message and puts it in the queue that the CLI script is watching. The CLI script receives it and then closes the queue.

When the CLI script does not get a response in 30 seconds it will exit with an error indicating the background daemon is (most likely) hung.

I tied all this into Monit. Monit can now check that the background daemon is running (via the pidfile and process list) and verify that it is actually still processing messages (by using the CLI tool to test that it responds to status requests)

Sander Marechal
A: 

Hello Sander Marechal,

can you tell me which phpclass you use for your cli-script communicationg with beanstald?

I use the beanstald class hosted on sourceforge, but my waiting background process use 50-80% cpu, while waiting for a job. Are there any other options oder classes?

Thanks for your help.

Kind regrads

Alexander

Alexander
Sander Marechal
Hello Sander,many thanks for your answer. That sound really good. I am not sure whats wrong on my System because i still have this high CPU usage with the patched file. Will try it tomorrow again :)Have a nice day.
Alexander