I have two servers - one Django, the other likely to be written in Python - and one is putting 'tasks' into a database and another is processing these tasks.
They share a database, but I want the processor to react quickly to new tasks rather than polling periodically.
Are there any straightforward ways for two Python servers to talk to one another, or does the task processor have to have web-hooks or something?
It feels there ought to be a blessed way to do this...
I tend to use polling. If the task table isn't that large it doesn't really involve that much overhead.
Otherwise you can implement a web service, or socket type connections.
You can use SOAPpy to start writing web service stuff, or just extends BaseHTTPServer or something like that to accept messages (HTTP requests) from Django. I do feel that might be more programming than it's worth, but then again, if the tasks only come infrequently it might be the neatest solution.
I would however run my home-build mini-server in some protected environment; only Django should be able to do HTTP requests on there, as it's not easy to build a secure web server.
EDIT
I just thought about Twisted. This may be the perfect network part for your server if you decide not to use a messaging queue (some twisted examples)
Look toward message brokers like ActiveMQ, RabbitMQ, ZeroMQ. They are designed to solve problems similar to what you've described.
I'm working on real-time MMORPG with server part written in Python and our daemons currently queue tasks to each other using ActiveMQ with STOMP protocol.
On low level message brokers keep socket connections to consumers, so it is more efficient than periodical polling.
SimpleXMLRPCServer.
See my answer here: http://stackoverflow.com/questions/2128266/network-programming-in-python/2129373#2129373
You could also use periodic polling (in case stuff gets lots) but xmlrpcserver should be fine for most of the work.