Hi, The system we are building is receiving data through the external feed. Our job is to distribute this data to multiple services, run the calculations and forward the results elsewhere - typical publisher-subscriber situation. What we need is a very low latency messaging. We don't need to persist the messages like MSMQ. Is RabbitMq ...
Hi, So I just setup celery and rabbitmq, created my user, setup the vhost, mapped the user to the vhost, and ran the celery daemon succesfully (or so I assume) (queuetest)corky@corky-server:~/projects/queuetest$ ./manage.py celeryd celery@corky-server v0.9.5 is starting. Configuration -> . broker -> amqp://celery@localhost:5672/ . qu...
I'd like to create a farm of processes that are able to OCR text. I've thought about using a single queue of messages which is read by multiple OCR processes. I would like to ensure that: each message in queue is eventually processed the work is more or less equally distributed an image will be parsed only by one OCR process An OCR pr...
I'd like to make sure that my message was delivered to a queue. To do so I'm adding the mandatory param to the basic_publish. What else should I do to receive the basic.return message if my message wasn't successfully delivered? I can't use channel.wait() to listen for the basic.return because when my message is successfully delivered...
Hey guys, I'm looking to implement RabbitMQ into my PHP application, and am using the php-amqp extension. My only question is this, how do I easily query to return the contents of the queue in PHP? php-amqp seems to not enable me to do this. If I am going wrong, please help me out here :) ...
I want to remove bindings in RabbitMQ without deleting the bound queue. I am using txAMQP with the 0.8 AMQP spec; it seems to be the only version that RabbitMQ supports but it has no unbind method. Oddly enough, a perusal through the source code suggests that RabbitMQ supports unbind, which leaves me rather confused. Can I unbind with ...
We have C++ Software that needs to send message via RabbitMQ broker, has somebody tried ActiveMQ CPP lib (CMS) to send/receive messages to RabbitMQ broker? ...
At work, we need to build a jobs server for things like sending emails, building PDFs, crunching some data, etc. Obviously, we'd like to build on some sort of generic queueing system. I'm familiar with Gearman, and that this is exact problem that it tries to solve: putting jobs on a queue where workers come to pick them up. However, I'...
Where is the best place initialize a Singleton in Rails? I am using the Carrot AMQP library in a Ruby on Rails app and I only want to initial the settings once and not on every task that is generated. I currently have it in my environment.rb and it seems to work but I am not entirely sure this is the best place. Is having Carrot initi...
I want to send a message whenever RabbitMQ creates or deletes a queue, exchange or binding. Is this feasible from a client API? What about as a plug-in? ...
I am designing a RabbitMQ based message processing system for use with various PHP based web applications. PHP daemons will be used to retrieve items from a queue and perform processing. The idea is for example, emailing hundreds of receipients, the message queue will do it asyncronously to the web application. My question is, with thi...
Hi, I'm looking for a general overview of best practice to implement a "job allocation" system like Gearman using RabbitMQ messaging system. So basically I'd have something reading the queue and forking? instances to run a job? Any insight is appreciated. ...
Hi, Does RabbitMQ support a scenario where a received message acknowledgement is sent on the DB transaction commit? Currently we send ack after DB transaction commit. If service fails inbetween, we'll get data duplication - service will get the same message again. Is there a pattern for this problem? Thanks! ...
Hello everyone. I need to set up a job/message queue with the option to set a delay for the task so that it's not picked up immediately by a free worker, but after a certain time (can vary from task to task). I looked into a couple of linux queue solutions (rabbitmq, gearman, memcacheq), but none of them seem to offer this feature out of...
There are many activities on an application that need things like: Send email, Post to twitter thumbnail an image, into several sizes call a cron to find connected relationships A good way to do these tasks is to write into an asynchronous queue on which operations are performed. What django application can be used to implement such...
We are getting close to releasing our new rails app and so far interest seems very strong we are a little worried about where bottle necks will be. One seems to be system e-mails on signup and in other situations. Is this correct? Should individual e-mails to users be sent asynchronously in the background? If so, what would be the be...
Is there an Erlang AMQP client library that isn't tied to the hip with RabbitMQ ? I know about http://hg.rabbitmq.com/rabbitmq-erlang-client/ but I would like something more decoupled... pain when it comes to packaging this stuff for a Debian repository, installation etc... ...
I'm using RabbitMQ on RHEL 5.3 using the Java client. I have 2 nodes (machines). Node1 is consuming messages from a queue on Node2 using the Java helper class QueueingConsumer. QueueingConsumer consumer = new QueueingConsumer(channel); channel.basicConsume("MyQueueOnNode2", noAck, consumer); while (true) { QueueingConsumer.Delivery...
Hi all, I'm looking for the right messaging system for my needs. Can you help me ? For now, there won't be a huge amount of data to process, but I don't want to be limited later ... The machines are not just web servers, so the messaging tool should be lightweight, even if processing is not very speed. When some data change on a serve...
How can I delete a regular Task or PeriodicTask in celery? ...