How can I display updating output of a slow script using Pylons?

Question

I am writing an application in Pylons that relies on the output of some system commands such as traceroute. I would like to display the output of the command as it is generated rather than wait for it to complete and then display all at once.

I found how to access the output of the command in Python with the answer to this question:

How can I perform a ping or traceroute in python, accessing the output as it is produced?

Now I need to find a way to get this information to the browser as it is being generated. I was planning on using jQuery's loadContent() to load the output of a script into a . The problem is that Pylons controllers use return so the output has to be complete before Pylons renders the page and the web server responds to the client with the content.

Is there any way to have a page display content as it is generated within Pylons or will this have to be done with scripting outside of Pylons?

Basically, I'm trying to do something like this: http://network-tools.com/default.asp?prog=trace&host=www.bbc.co.uk

Answer 1

pexpect will let you get the output as it comes, with no buffering.

To update info promptly on the user's browser, you need javascript on that browser sending appropriate AJAX requests to your server (dojo or jquery will make that easier, though they're not strictly required) and updating the page as new responses come -- without client-side cooperation (and JS + AJAX is the simplest way to get that cooperation), there's no sensible way to do it on the server side alone.

So the general approach is: send AJAX query from browser, have server respond as soon as it has one more line, JS on the browser updates contents then immediately sends another query, repeat until server responds with an "I'm all done" marker (e.g. an "empty" response may work for that purpose).

Answer 2

I haven't tried it with pylons, but you could try to show the output of the slow component in an iframe on the page (using mime type text/plain) and yield each chunk to the iframe as it is generated. For fun I just put this together as a WHIFF demo. Here is the slowly generated web content wsgi application:

import time

def slow(env, start_response):
    start_response("200 OK", [('Content-Type', 'text/plain')])
    return slow_generator()

def slow_generator():
    yield "slowly generating 20 timestamps\n"
    for i in range(20):
        yield "%s: %s\n" % (i, time.ctime())
        time.sleep(1)
    yield "done!"

__wsgi__ = slow

This file is deployed on my laptop at: http://aaron.oirt.rutgers.edu/myapp/root/misc/slow.

Here is the WHIFF configuration template which includes the slow page in an iframe:

{{env whiff.content_type: "text/html"/}}

Here is an iframe with slowly generated content:
<hr>
<iframe frameborder="1" height="300px" width="300px" scrolling="yes"
style="background-color:#99dddd;"
src="slow"
></iframe>
<hr>
Isn't that cool?

This is deployed on my laptop at http://aaron.oirt.rutgers.edu/myapp/root/misc/showSlowly.

hmmm. I just tried the above link in safari and it didn't work right... apparently there are some browser differences... Seems to work on Firefox at least...

Answer 3

You may want to look at this faq entry. Then with JS, you always clear the screen before writing new stuff.