What's a good way to connect the synchronous http request/response model with an asynchronous queue based model? When the user's HTTP request comes it generates a work request that goes onto a queue (beanstalkd in this case). One of the workers picks up the request, does the work, and prepares a response. The queue model is not request...
I have an application in Python 2.5 that listens to a beanstalk queue. It works fine on all machines I tested so far, except from my newly acquired MacBook Pro. On that computer, when I try to run it I get this error: Traceback (most recent call last): File "jobs.py", line 181, in <module> Jobs().start() File "jobs.py", line 1...
i am using Solaris 10 OS(x86). i installed beanstalkd and it starts fine by using command "beanstalkd -d -l hostip -p 11300". i have Python 2.4.4 on my system i installed YAML and beanstalkc python libraries to connect beanstalkd with python my problem is when i try to write some code: import beanstalkc beanstalk = beanstalkc.Connectio...
Hello, would you recommend using beanstalkd [ http://kr.github.com/beanstalkd/ ] for asynchronous processing in PHP ? I need some feedback/comments pros-cons, from anyone that has used this lib. Thanks, ...
So what I like about beanstalkd: small, lightweight, has priorities for messages, has a great set of clients, easy to use. What I dislike about beanstalkd: the lack of authentication menaing if you can connect to the port you can insert messages into it. So my thoughts are to either firewall it to trusted systems (which is a pain to ...
I have a website written in PHP (CakePHP) where certain resource intensive tasks are handled by a background process. This is done through the Beanstalkd message queue. I need some way to retrieve the status of that background process so I can monitor it with Monit. The background process is a CakePHP Shell (just a PHP CLI script) that ...
We have a large project coming up soon with quite a lot of media processing (Images, Video) as well email output etc, the sort of stuff normally we'd put into a table called "email_queue" and we use a cron to run a script process the queue in the table. I have been reading a lot on Message Queue systems like beanstalkd, and have even se...
Actually, I would like to use this for logging. I want to put a dictionary into beanstalkd. Everytime someone goes into my website, I want to put a dictionary into beanstalkd, and then every night, I want a script that will get all the jobs and stick them in the database. THis will make it fast and easy. ...
I'm using the Perl client of beanstalkd. I need a simple way to not enqueue the same work twice. I need something that needs to basically wait until there are K elements, and then groups them together. To accomplish this, I have the producer: insert item(s) into DB insert a queue item into beanstalkd And the consumer: while ( 1 ) ...
Does anyone know of an app that can monitor a beanstalkd queue? I'm looking for something that shows stats on tubes and jobs, and allows you to inspect the details. I'm not really picky about language/platform, just want to know if there's something out there before I write my own. ...
Hello everyone. I need to set up a job/message queue with the option to set a delay for the task so that it's not picked up immediately by a free worker, but after a certain time (can vary from task to task). I looked into a couple of linux queue solutions (rabbitmq, gearman, memcacheq), but none of them seem to offer this feature out of...
I'm trying to figure out the most efficient way to running a pretty hefty PHP task thousands of times a day. It needs to make an IMAP connection to Gmail, loop over the emails, save this info to the database and save images locally. Running this task every so often using a cron isn't that big of a deal, but I need to run it every min...
I have tried on two different servers to get beanstalkd up and running and do a couple tests (locally on MacOSX compiled from source, and on a CentOS server installed with yum) I can get the server running either with sudo beanstalkd -d -p 11300 or sudo beanstalkd -p 11300 & I then tried using the php lib and it just froze. Conne...
Does anyone know of any interface to administer beanstalkd? What I'm looking for vaguely is something like PhpMyAdmin for MySQL. Doesn't have to be PHP or web-based, could be any interface, GUI ..etc ...
We are going to implement a fairly complex new service in an asynchronous way, introducing asynchronous processing as well. For this, we have selected beanstalkd message queue service, as it fits best for our needs. The problem is that beanstalk seems to miss client API in C, at least I have failed to find it. I do not want to invent t...
beanstalkd is playing tricks on me. As you can see from the following series of commands, there are ready jobs, but I cant peek, delete, or do anything with them. watch ab_similar_sync_node WATCHING 1 stats-tube ab_similar_sync_node OK 272 --- name: ab_similar_sync_node current-jobs-urgent: 0 current-jobs-ready: 8521 current-jobs-reser...
I've been trying to find a decent beanstalkd GUI with no luck at all. I decided to write my own. I'm thinking of doing something totally AJAXy where you can see everything in one page, list tubes, see jobs in a tube, flash charting maybe ...etc I'm terrible with UI, so I was wondering if anyone can recommend some examples that are relev...
Hi, all I am trying to use beanstalk for queuing a large number of periodic tasks (for example, tasks need processed every N minutes), for each task, if the last queued job is not completed (not reserved, i mean) when current job to be added, the last queued job should be replaced with current job, in other words, only the latest queued...
Here's how my servers (in Amazon EC2) would look like: Server 1 Server 2 Server 3 __________________________ _____________________ _______________ | Cloud Monitor Daemon | | Memcached daemon | | beanstalkd | | | | Memcached daemon | ____...
I'm currently trying to test out beanstalkd I have installed ubuntu (on vmware workstation) Running ifconfig, I can see the ip address is 192.168.0.190 I start beanstalkd on port 11300 From my host machine (windows) I try to telnet to the ipaddress / port, however it tells me I can't connect... What's going wrong? (ps- im a linux n...