Send a "304 Not Modified" for images stored in the datastore

Question

I store user-uploaded images in the Google App Engine datastore as db.Blob, as proposed in the docs. I then serve those images on /images/<id>.jpg.

The server always sends a 200 OK response, which means that the browser has to download the same image multiple time (== slower) and that the server has to send the same image multiple times (== more expensive).

As most of those images will likely never change, I'd like to be able to send a 304 Not Modified response. I am thinking about calculating some kind of hash of the picture when the user uploads it, and then use this to know if the user already has this image (maybe send the hash as an Etag?)

I have found this answer and this answer that explain the logic pretty well, but I have 2 questions:

1. Is it possible to send an Etag in Google App Engine?
2. Has anyone implemented such logic, and/or is there any code snippet available?

Answer 1

Bloggart uses this technique. Have a look at this blog post.

class StaticContentHandler(webapp.RequestHandler):
  def output_content(self, content, serve=True):
    self.response.headers['Content-Type'] = content.content_type
    last_modified = content.last_modified.strftime(HTTP_DATE_FMT)
    self.response.headers['Last-Modified'] = last_modified
    self.response.headers['ETag'] = '"%s"' % (content.etag,)
    if serve:
      self.response.out.write(content.body)
    else:
      self.response.set_status(304)

  def get(self, path):
    content = get(path)
    if not content:
      self.error(404)
      return

    serve = True
    if 'If-Modified-Since' in self.request.headers:
      last_seen = datetime.datetime.strptime(
          self.request.headers['If-Modified-Since'],
          HTTP_DATE_FMT)
      if last_seen >= content.last_modified.replace(microsecond=0):
        serve = False
    if 'If-None-Match' in self.request.headers:
      etags = [x.strip('" ')
               for x in self.request.headers['If-None-Match'].split(',')]
      if content.etag in etags:
        serve = False
    self.output_content(content, serve)

Answer 2

There might be a simpler solution here. This requires that you never overwrite the data associated with any identifier, e.g. modifying the image would create a new id (and hence a new URL).

Simply set the Expires header from your request handler to the far future, e.g. now + a year. This would result in clients caching the image and not asking for an update until that time comes.

This approach has some tradeoffs, like ensuring new URLs are embedded when images are modified, so you have to decide for yourself. What jbochi is proposing is the other alternative that puts more logic into the image request handler.

Answer 3

By the way, thanks to webob, webapp.RequestHandler provides easy way to check If-None-Match.

if etag in self.request.if_none_match:
    pass # do something