Server-side rendering using blender and twisted (python)

Question

Hi everyone,

The project I am working on at the moment basically takes in an image and then renders a video using blender from the command line. At the moment I am using Twisted to deal with the requests but there is certainly something that I am doing wrong as it is not working how I would like it to. You can see the jist of the program here (I have stripped out anything unnecessary).

The blender render is done by spawning a subprocess (I am aware Twisted can handle processes) along with a python script to configure the render and use the image provided as a texture.

The program needs to be able to handle as many connections as possible. At the moment the subprocess does one render at a time but ideally it would check CPU/number of parallel renders and adjust the number to the optimum. Each render is custom to the user so once a users render is complete they should get their render back (an avi file).

My question is: Is Twisted the right choice for this? Are there any other options? If not, is my implementation of the system flawed? I would appreciate any thoughts or opinions on this!

Answer 1

Is Twisted the right choice for this? - Perhaps.

Are there other options? - Yes.

If not, is my implementation of the system flawed? - Yes. It looks to me that your subprocess call is blocking: p.wait()

It is possible to do what it sounds like you're trying to do in Twisted, but you are a very long way from it.

You need a rate-controlled, asynchronous task queue with a web frontend.

What you've got is single page on a single threaded 'site' that doesn't return any html until the submitted job is finished.

This is possible in twisted. However it's probably easier to implement using django + celery.

Django: http://www.djangoproject.com/
Celery: http://ask.github.com/celery/getting-started/introduction.html

And a tutorial for a similar purpose:
http://webcookies.org/blog/2009/09/10/rabbitmq-celery-and-django/