Google app engine key value error

Question

I am writing a google app engine app and I have this key value error upon requests coming in

from the backtrace I just access and cause the key error

self.request.headers

entire code snippet is here, I just forward the headers unmodified

     response = fetch( "%s%s?%s" % (
                                        self.getApiServer() ,
                                        self.request.path.replace("/twitter/", ""),
                                        self.request.query_string
                                    ),
                        self.request.body,
                        method,
                        self.request.headers,
                    )

and get method handling the request calling proxy()

# handle http get   
def get(self, *args):
    parameters = self.convertParameters(self.request.query_string)
    # self.prepareHeader("GET", parameters)
    self.request.query_string = "&".join("%s=%s" % (quote(key) , quote(value)) for key, value in parameters.items())
    self.proxy(GET, *args)
def convertParameters(self, source):
    parameters = {}
    for pairs in source.split("&"):
        item = pairs.split("=")
        if len(item) == 2:
            parameters[item[0]] = unquote(item[1])
    return parameters

the error back trace:

'CONTENT_TYPE'
Traceback (most recent call last):
  File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", line 513, in __call__
    handler.post(*groups)
  File "/base/data/home/apps/waytosing/1.342850593213842824/com/blogspot/zizon/twitter/RestApiProxy.py", line 67, in post
    self.proxy(POST, *args)
  File "/base/data/home/apps/waytosing/1.342850593213842824/com/blogspot/zizon/twitter/RestApiProxy.py", line 47, in proxy
    self.request.headers,
  File "/base/python_runtime/python_lib/versions/1/google/appengine/api/urlfetch.py", line 240, in fetch
    allow_truncated, follow_redirects)
  File "/base/python_runtime/python_lib/versions/1/google/appengine/api/urlfetch.py", line 280, in make_fetch_call
    for key, value in headers.iteritems():
  File "/base/python_runtime/python_dist/lib/python2.5/UserDict.py", line 106, in iteritems
    yield (k, self[k])
  File "/base/python_runtime/python_lib/versions/1/webob/datastruct.py", line 40, in __getitem__
    return self.environ[self._trans_name(item)]
KeyError: 'CONTENT_TYPE'

Any idea why it happens or is this a known bug?

Answer 1

Edit: indeed, looking at the error more carefully, it doesn't seem to be related to convertParameters, as the OP points out in the comments. I'm retiring this answer.

I'm not entirely sure what you mean by "just forward the headers unmodified", but have you taken a look at self.request.query_string before and after you call convertParameters? More to the point, you're leaving out any (valid) GET parameters of the form "key=" (that is, keys with empty values).

Maybe your original query_string had a value like "CONTENT_TYPE=", and your convertParameters is stripping it out.

Answer 2

Known issue http://code.google.com/p/googleappengine/issues/detail?id=3427 and potential workarounds here http://code.google.com/p/googleappengine/issues/detail?id=2040

Answer 3

This looks weird. The docs mention that response "Headers objects do not raise an error when you try to get or delete a key that isn't in the wrapped header list. Getting a nonexistent header just returns None". It's not clear from the request documentation if request.headers are also objects of this class, but even they were regular dictionaries, iteritems seems to be misbehaving. So this might be a bug.

It might be worth inspecting self.request.headers, before calling fetch, and see 1) its actual type, 2) its keys, and 3) if trying to get self.request.headers['CONTENT_TYPE'] raises an error then.

But, if you simply want to solve your problem and move forward, you can try to bypass it like:

if 'CONTENT_TYPE' not in self.request.headers:
    self.request.headers['CONTENT_TYPE'] = None

(I'm suggesting setting it to None, because that's what a response Header object should return on non-existing keys)

Answer 4

Here's my observation about this problem: When the content-type is application/x-www-form-urlencoded and POST data is empty (e.g. jquery.ajax GET, twitter's favorite and retweet API...), the content-type is dropped by Google appengine. You can add:

self.request.headers.update({'content-type':'application/x-www-form-urlencoded'}) 

before urlfetch.