Update HttpResponse Every Few Seconds.

Question

My application in Django can create some very big SQL queries. I currently use a HttpRequest object, for the data I need, then a HttpResponse, to return what I want to show the User.

Obviously, I can let the User wait for a minute whilst these many sets of queries are being executed and extracted from the database, then return this monolothic HTML page.

Ideally, I'd like to update the page when I want, something like:

For i,e in enumerate(example):

    Table.objects.filter(someObjectForFilter[i]).

    #Return the object to the page.
    #Then Loop again, 'updating' the response after each iteration.

Is this possible?

Answer 1

Your approach is a bit flawed. You have a few different options.

The first is probably the easiest - use AJAX and HTTPRequest. Have a series of these, each of which results in a single Table.objects.filter(someObjectForFilter[i]).. As each one finishes, the script completes and returns the results to the client. The client updates the UI and initiates the next query via another AJAX call.

Another method is to use a batch system. This is a bit heftier, but probably a better design if you're going for real "heavy lifting" in the database. You'll need to have a batch daemon running (a cron probe works just fine for this) scanning for incoming tasks. The user wants to perform something, so their request submits this task (it could simply be a row in a database with their paramters). The daemon grabs it, processes it completely offline - perhaps even by a different machine - and updates the task row when it's complete with the results. The client can then refresh periodically to check the status of that row, via traditional or AJAX methods.

Answer 2

I discovered recently that an HttpResponse can be a generator:

def myview(request, params):
    return HttpResponse(mygenerator(params))


def mygenerator(params):
    for i,e in enumerate(params):
        yield '<li>%s</li>' % Table.objects.filter(someObjectForFilter[i])

This will progressively return the results of mygenerator to the page, wrapped in an HTML <li> for display.