Returning Database Blobs in TurboGears 2.x / FCGI / Lighttpd extremely slow

Question

Hey everyone,

I am running a TG2 App on lighttpd via flup/fastcgi. We are reading images (~30kb each) from BlobFields in a MySQL database and return those images with a custom mime type via a controller method. Caching these images on the hard disk makes no sense because they change with every request, the only reason we cache these in the DB is that creating these images is quite expensive and the data used to create the images is also present in plain text on the website.

Now to the problem itself:

When returning such an image, things get extremely slow. The code runs totally fine on paster itself with no visible delay, but as soon as its running via fcgi/lighttpd the described phenomenon happens.

I profiled the method of my controller that returns my blob, and the entire method runs in a few miliseconds, but when "return" executes, the entire app hangs for roughly 10 seconds.

We could not reproduce the same error with PHP on FCGI. This only seems to happen with Turbogears or Pylons.

Here for your consideration the concerned piece of source code:

@expose(content_type=CUSTOM_CONTENT_TYPE)
def return_img(self, img_id):
    """ Return a DB persisted image when requested """

    img = model.Images.by_id(img_id) #get image from DB
    response.headers['content-type'] = 'image/png'
    return img.data # this causes the app to hang for 10 seconds

Answer 1

I've no clue, really, but seeing as there are no answers here, I'll try a wild guess.

Perhaps

response.headers['content-length'] = len(img.data)

would help?