Looking for python lib to manage remote tasks

Question

Hi, I have server with django on it, this server runs some manage.py commands and update database. Now I need to move some of this tasks to different servers. I don't want to allow remote db access and need some tool\lib to be able to start task on remote servers by main server's command and update tasks code/add new tasks. I have ssh access to every server, all servers run under debian and all code in python.

I was thiking about creating my own xmpp based solution(server sends messages to slave servers with commands to execute, like "update task", "run task"), or maybe some low-level ssh based solution where main server logs to slave servers and executes bash commands. But I would be happy to hear any advices.

Answer 1

Sounds like celery will be right up your street:

Celery is a task queue/job queue based on distributed message passing

They have a whole Django-based version, so it should be just what you're looking for.

Answer 2

Fabric is the standard tool for remote management.

You would have to combine that with some cron task calling the fabric commands.

Answer 3

Depending on your needs you could use ssh to create a tunnel from the current server to the db on the db server:

ssh -N -f -L 3307:localhost:3306 myname@myhost