Streaming the result of a command back to the browser using Twisted and Comet

Question

I'm writing an application that streams the output (by this I mean both sys.stdout and sys.stderr) of a python script excited on the server, in real time to the browser.

The users on the site will be allowed to select the script to run, excite and kill their chosen script, and change some parameters, so I will need a different thread per user on the site (user A can start, stop and change a script, whilst user B can do the same with a different script).

I know I need to use comet for the web clients, and seeing as the rest of the project is written in python, I'd like to use twisted for the server, however I'm not really sure of what I need to do next!

There are a daunting number of options (Divmod Mantissa, Divmod Nevow, twisted.web, STOMP, etc), and some are better documented that others, making the whole thing rather tricky!

I have a working demo using stompservice on orbited, using Orbited.TCPSocket for the javascript side of things, however I'm starting to think that STOMPs channel model isn't going to work for multithreading, multi-running scripts (unless I open a new channel per run, but that seems like the wrong use of the channel model).

Can anyone point me in the right direction, or some sample code I can learn from?

Thanks!

Answer 1

An example of serving for long-polling clients with Twisted is slosh. This might not be what you want, but because it's not a large framework, it can help you figure out how to use Twisted.

Answer 2

Nevow Athena is a framework specifically for AJAX and COMET applications and in theory is exactly the sort of thing you are looking for.

However, I am not sure that it is well used or supported at this time - looking at mailing list traffic and google search results suggests that it may not be.

There are a couple of tutorials you could look at to help you decide on it:

• one on the 'official' site: http://divmod.org/trac/wiki/DivmodNevow/Athena/Tutorials/LiveElement

• and one other that I found: http://divmodsphinx.funsize.net/nevow/chattutorial/part01/index.html

The code for the latter seems to be included in the Nevow distribution when you download it under /doc/listings/partxx (I think...)