How do I display real-time python script output on a website?

Question

I have a Python script that outputs something every second or two, but takes a long while to finish completely. I want to set up a website such that someone can directly invoke the script, and the output is sent to the screen while the script is running.

I don't want the user to wait until the script finishes completely, because then all the output is displayed at once. I also tried that, and the connection always times out.

I don't know what this process is called, what terms I'm looking for, and what I need to use. CGI? Ajax? Need some serious guidance here, thanks!

If it matters, I plan to use Nginx as the webserver.

Answer 1

The solution is to flush the output buffer at select points in the script's execution - I've only ever done this in PHP via flush() but this looks like the Python equivalent:

cgiprint() also flushes the output buffer using sys.stdout.flush(). Most servers buffer the output of scripts until it's completed. For long running scripts, 8 buffering output may frustrate your user, who'll wonder what's happening. You can either regularly flush your buffer, or run Python in unbuffered mode. The command-line option to do this is -u, which you can specify as #!/usr/bin/python -u in your shebang line.

Answer 2

nginx doesn't support CGI, so you will need to use fastcgi or wsgi

Alternatively you could use something like webpy and proxy to it through nginx

Answer 3

First of all - your script must output header:

Connection: Keep-Alive

Because browser must know that it will have to wait.

And your script must output data without buffering. And stackoverflow has already answered this question.

Answer 4

From http://tools.cherrypy.org/wiki/Comet "Wikipedia describes Comet as, "a neologism to describe a web application model in which a long-held HTTP request allows a web server to push data to a browser, without the browser explicitly requesting it.". In other words, Comet is that asynchronous Javascript mojo you need to build really fancy AJAX applications.

The following code demonstrates how to write a Comet application using CherryPy and jQuery. It is a web interface into the console ping command. The ping command was chosen for this example because it will run indefinitely if given no arguments. Executing never-ending commands is usually a big no-no when it comes to web application programming but with CherryPy we can handle this quite easily:" SEE LINK

I am trying to do the exact same thing. So far I am using cherrypy because it is easy to integrate with my script.