I'm attempting to follow the celery tutorial, but I run into a problem when I run python manage.py celeryd: my RabbitMQ server (installed on a virtual machine on my dev box) won't let my user login. I get the following on my Django management console: [ERROR/MainProcess] AMQP Listener: Connection Error: Socket closed. Trying again in 2...
I've looked into PeriodicTask, but the examples only cover making it recur. I'm looking for something more like cron's ability to say "execute this task every Monday at 1 a.m." ...
I'm having a problem using imaplib on python 2.6 with the latest django svn. I want to download imap emails in a queue (using celeryd). I'm able to connect/download emails from the command line, but when i offload the task through django to celeryd i get this error: "SSLError: [Errno 1] _ssl.c:1325: error:1408F10B:SSL routines:SSL3_GET_R...
I have a Django project and I'm trying to use Celery to submit tasks for background processing ( http://ask.github.com/celery/introduction.html ). Celery integrates well with Django and I've been able to submit my custom tasks and get back results. The only problem is that I can't find a sane way of performing custom initialization in t...
Hi, So I just setup celery and rabbitmq, created my user, setup the vhost, mapped the user to the vhost, and ran the celery daemon succesfully (or so I assume) (queuetest)corky@corky-server:~/projects/queuetest$ ./manage.py celeryd celery@corky-server v0.9.5 is starting. Configuration -> . broker -> amqp://celery@localhost:5672/ . qu...
I have python server based on django and celery. Each time computer restarts, apache2 starts so my server is working, BUT I have to restart celery manually (going to my project directory and executing "python manage.py celeryd"). What is the correct solution for production? I mean, is there a possibility to start celery as daemon? Here...
There are many activities on an application that need things like: Send email, Post to twitter thumbnail an image, into several sizes call a cron to find connected relationships A good way to do these tasks is to write into an asynchronous queue on which operations are performed. What django application can be used to implement such...
Hi everyone, I have pretty standard Django+Rabbitmq+Celery setup with 1 Celery task and 5 workers. Task uploads the same (I simplify a bit) big file (~100MB) asynchronously to a number of remote PCs. All is working fine at the expense of using lots of memory, since every task/worker load that big file into memory separatelly. What I ...
How can I delete a regular Task or PeriodicTask in celery? ...
For a Python/Django/Celery based deployment tool, we have the following setup: We currently use the default Celery setup. (One queue+exchange called "celery".) Each Task on the queue represents a deployment operation. Each task for an environment ends with a synchronisation phase that potentially takes (very) long. The following spec...
Hello to all! I am writing a small Django application and I should be able to create for each model object its periodical task which will be executed with a certain interval. I'm use for this a Celery application, but i can't understand one thing: class ProcessQueryTask(PeriodicTask): run_every = timedelta(minutes=1) def run(self...
I'm posting this question (and answer) so if anybody else has this problem in the future, you'll be able to google it. If you are trying to run celeryd in Django like so: python manage.py celeryd You can receive the following error immediately after it has started: celery@eric-desktop-dev has started. Traceback (most recent call las...
I have a package named jiva_tasks, which I'm trying to import via celery (using the CELERY_IMPORTS attribute of celeryconfig. The import statement that celery is using is this: __import__(module, [], [], ['']) Oddly enough, when this syntax is used, the module gets imported twice, once as jiva_tasks and another time as jiva_tasks. (w...
We're working on an application that supports AMQP for queuing. Some of our clients are using Websphere MQ. I'm just wondering at a high level how interchangeable these two protocols are in terms of functionality. I'm using celery, which should allow me to abstract out the lower-level stuff as long as I can write a Websphere MQ backen...
How can I get the task_id value for a task from within the task? Here's my code: from celery.decorators import task from django.core.cache import cache @task def do_job(path): "Performs an operation on a file" # ... Code to perform the operation ... cache.set(current_task_id, operation_results) The idea is that when I c...
I installed django-celery and I tried to start up the worker server but I get an OSError that a function isn't implemented. I'm running CentOS release 5.4 (Final) on a VPS: . broker -> amqp://guest@localhost:5672/ . queues -> . celery -> exchange:celery (direct) binding:celery . concurrency -> 4 . loader -> djce...
I've built a Django web application and some Django-Piston services. Using a web interface a user submits some data which is POSTed to a web service and that web service in turn uses Django-celery to start a background task. Everything works fine in the development environment using manage.py. Now I'm trying to move this to production o...
I have a Django project that uses Celery for running asynchronous tasks. I'm doing my development on a Windows XP machine. Starting my Django server (python manage.py runserver 80) works fine, but attempting to start the Celery Daemon (python manage.py celeryd start) fails with the following error: ImportError: Could not import settin...
I want to have some place where I can watch all the pendings tasks. I'm not talking about the registered functions/classes as tasks, but the actual scheduled jobs for which I could display: name, task_id, eta, worker, etc. Using Celery 2.0.2 and djcelery, I found `inspect' in the documentation. I tried: from celery.task.control import...
Not talking about the delay method. I want to be able to get a task, given it's task_id and change it's ETA on the fly, before it is executed. For now I have to cancel it, and re-schedule one. Troublesome if the scheduled process involve a lot of stuff. ...