views:

716

answers:

3

I've been looking into non-blocking servers for python (tornado, twisted etc) but a lot of the benefits seems to be lost if there's no non-blocking connection to the database. Does anyone know if there are any projects that take care of this? (by non-blocking a la node.js)

Edit: Clarified my question

A: 

This may be stupid but isn't async troublesome in general (in python) due the the global interpreter lock?

extraneon
No. Asynchronous generally means "callback when results are available". This is orthogonal to the GIL. You could imagine an asynchronous, thread-based computation (ie, CPU-bound) library which *would* have problems because of the GIL - but this is due to the use of threads for multiple CPU-bound tasks. There are plenty of other things (for example, talking to a MySQL server over a socket) which are not multithreaded CPU-bound tasks.
Jean-Paul Calderone
+1  A: 

The way you do that is by spawning database queries in a separate thread. With Twisted you can use deferToThread() or deferToThreadPool() (see the API docs[1]).

Antoine P.
Additionally, twisted.enterprise.adbapi provides a DB-API-like API on top of `deferToThreadPool` specifically to make it easier to access MySQL and other rdbms in a Twisted-based application.
Jean-Paul Calderone
+3  A: 

You can use Twisted's ADBAPI to wrap a synchronous DBAPI implementation.

E.g.:

from twisted.internet import reactor
from twisted.enterprise import adbapi

def result(rows):
    for row in rows:
        print row

    reactor.stop()

def fail(err):
    err.printDetailedTraceback()
    reactor.stop()

pool = adbapi.ConnectionPool('sqlite3', 'animals.db')
d = pool.runQuery('SELECT * FROM animals', ())
d.addCallbacks(result, fail)
reactor.run()
daf