Properties of a PHP Worker Pool for Background Processing with Gearman

Question

Hello,

I did some reading about Gearman and i am wondering if it can be used as a worker pool for background processing. I am interested in a PHP worker pool with the following properties:

A. Workers should be able to run concurrently

That is, i should be able to have a worker pool and i am expecting gearman to dispatch multiple workers in separate threads. I might need a worker pool manager that manages the number of workers waiting for an incoming job.

Can gearman provide this? Have i missed the point?

any help appreciated.

Answer 1

This sounds like an ideal use case for gearman, actually.

Essnetially you can have 1 or more gearman servers where jobs are sent to, and then you can have 1 or more workers. The workers will connect to the gearman server(s) and ask for jobs. The workers can be on the same machines as the servers or on completely different machines. You could easily have 20 workers all running at the same time, each working on different jobs fetched from gearman servers.

I recommend starting with this presentation to get a good idea of what you can do with gearman:

http://assets.en.oreilly.com/1/event/27/Gearman_%20Build%20Your%20Own%20Distributed%20Platform%20in%203%20Hours%20Presentation.pdf