Debug/Monitor middleware for python wsgi applications

Question

I'm searching a wsgi middleware which I can warp around a wsgi applications and which lets me monitor incoming and outgoing http requests and header fields.

Something like firefox live headers, but for the server side.

Answer 1

That shouldn't be too hard to write yourself as long as you only need the headers. Try that:

import sys

def log_headers(app, stream=None):
    if stream is None:
        stream = sys.stdout
    def proxy(environ, start_response):
        for key, value in environ.iteritems():
            if key.startswith('HTTP_'):
                stream.write('%s: %s\n' % (key[5:].title().replace('_', '-'), value))
        return app(environ, start_response)
    return proxy

Answer 2

The middleware

from wsgiref.util import request_uri
import sys

def logging_middleware(application, stream=sys.stdout):
    def _logger(environ, start_response):
        stream.write('REQUEST\n')
        stream.write('%s %s\n' %(
            environ['REQUEST_METHOD'],
            request_uri(environ),
        ))

        for name, value in environ.items():
            if name.startswith('HTTP_'):
                stream.write('    %s: %s\n' %(
                    name[5:].title().replace('_', '-'),
                    value,
                ))
        stream.flush()
        def _start_response(code, headers):
            stream.write('RESPONSE\n')
            stream.write('%s\n' % code)
            for data in headers:
                stream.write('    %s: %s\n' % data)
            stream.flush()
            start_response(code, headers)
        return application(environ, _start_response)
    return _logger

The test

def application(environ, start_response):
    start_response('200 OK', [
        ('Content-Type', 'text/html')
    ])
    return ['Hello World']

if __name__ == '__main__':
    logger = logging_middleware(application)
    from wsgiref.simple_server import make_server
    httpd = make_server('', 1234, logger)
    httpd.serve_forever()

See also the werkzeug debugger Armin wrote, it's usefull for interactive debugging.

Answer 3

If you want Apache-style logs, try paste.translogger

But for something more complete, though not in a very handy or stable location (maybe copy it into your source) is wsgifilter.proxyapp.DebugHeaders

And writing one using WebOb:

import webob, sys
class LogHeaders(object):
    def __init__(self, app, stream=sys.stderr):
        self.app = app
        self.stream = stream
    def __call__(self, environ, start_response):
        req = webob.Request(environ)
        resp = req.get_response(self.app)
        print >> self.stream, 'Request:\n%s\n\nResponse:\n%s\n\n\n' % (req, resp)
        return resp(environ, start_response)

Answer 4

The mod_wsgi documentation provides various tips on debugging which are applicable to any WSGI hosting mechanism and not just mod_wsgi. See:

http://code.google.com/p/modwsgi/wiki/DebuggingTechniques

This includes an example WSGI middleware that captures request and response.

Answer 5

My WebCore project has a bit of middleware that logs the entire WSGI environment (thus Beaker sessions, headers, etc.) for the incoming request, headers for outbound responses, as well as performance information to a MongoDB database. Average overhead is around 4ms.

The module has been removed from the core package, but hasn’t yet been integrated into its own. The current version as of this answer is available in the Git history:

http://github.com/GothAlice/WebCore/blob/cd1d6dcbd081323869968c51a78eceb1a32007d8/web/extras/cprofile.py