I am looking for a way to serve HTTP (and do HTTP requests) in an asynchronous, non-blocking fashion. This seems to be hard to do when you’ve decided on Stackless Python 3.1 (also see here for docs) as i did.
There are some basic examples, like the pretty informative and detailed article How To Use Linux epoll with Python, and there is a a Google code project named stacklessexamples which contains some valuable information (but no Python 3.x compatible code).
So, after many days of doing research on the web and trying to put together the pieces i’ve found so far: does anyone know of a fairly usable asynchronous HTTP library? It doesn’t have to be WSGI-compliant (I am not interested in that).
The server part should be able to serve multiple non-blocking HTTP requests (and possibly do the basics of HTTP header processing); the HTTP client part should be able to retrieve, in a non-blocking way, web content via HTTP requests (also doing basic header processing, but no fancy stuff like authorization or so).
My research so far has shown me that non-blocking HTTP
is the only way that makes sense in a stackless, cooperatively scheduled environment;
is feasible in Stackless Python 3 by virtue of the standard library’s
select epoll(introduced in Py2.6; some solutions preferlibevent, but that means another hurdle as the pyevent project seems to have stopped developing at Py2.5);is sadly still not a household item, with most people relying on blocking HTTP.
The way it looks like now, i would have to learn the basics of socket programming and roll my own HTTP server/client library. I still shy away from that task as i have very little background in that area and am bound to ‘repeat history’ that way.
I would be very happy about any relevant pointers. I prefer very much solutions that make use of select.epoll; i seem to remember it is much more scalable that the older asyncore (but maybe someone has more precise data on this). As a minimum requirement, solutions should run on Ubuntu 9.10.