Problems PUTting binary data to Django

Question

I am trying to build a RESTful api with Django to share mp3s -- right up front: it's a toy app, never going into production, so it doesn't need to scale or worry (I hope) about copyright devils. My problem now is that I have a Django view that I want to be the endpoint for HTTP PUT requests. The headers of the PUT will contain the metadata, and the body will exclusively be the binary.

Here's the actual view that I am (trying) to hit. Please note that logging indicates that control flow never enters the put() method, which I believe is correct, if not especially robust:

class UserSong(RESTView):
    logging.debug('entering UserSong.put')
    def put(self, request, username=''):

        if request.META['Content-Type'] != 'octet/stream':
            raise Http400() 

        title = request.META['X-BD-TITLE'] if 'X-BD-TITLE' in request.META else 'title unknown'
        artist = request.META['X-BD-ARTIST'] if 'X-BD-ARTIST' in request.META else 'artist unknown' 
        album = request.META['X-BD-ALBUM'] if 'X-BD-ALBUM' in request.META else 'album unknown' 
        song_data = b6decode(request.raw_post_data)

        song = Song(title=title, artist=artist, playcount=playcount, is_sample=is_sample, song_data=song_data, album=album)
        song.save()

        return HttpResponse('OK', 'text/plain' , 201)

def __call__(self, request, *args, **kwargs):
    logging.basicConfig(filename=LOGFILE,level=logging.DEBUG)
    try:  
        if request.method == 'DELETE':
            return self.delete(request, *args, **kwargs)  
        elif request.method == 'GET':
            return self.get(request, *args, **kwargs)
        elif request.method == 'POST':
            return self.post(request, *args, **kwargs)
        elif request.method == 'PUT':
            return self.put(request, *args, **kwargs)
    except:
        raise Http404()

In testing this, I was able to get unittests to pass using Django's unittesting framework, but I do not trust that it was accurately mimicking Real Life. So, I cracked open httplib, and constructed a PUT my own self. This is that code, which I executed interactively:

>>>method = 'PUT'
>>>url = 'accounts/test/songs/'
>>>f = open('/Users/bendean/Documents/BEARBOT.mp3')
>>>data = f.read()
>>>body = data
>>>headers = {'X-BD-ARTIST' : 'BEARBOT' , 'X-BD-ALBUM':'','X-BD-TITLE':'LightningSPRKS'}
>>>headers['CONTENT-TYPE'] = 'octet/stream'
>>>import httplib
>>>c = httplib.HTTPConnection('localhost:8000')
>>>c.request(method, url, body, headers)

the response I get is not pretty

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File  "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py", line 880, in request
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py", line 914, in _send_request
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py", line 719, in send
  File "<string>", line 1, in sendall
 error: [Errno 54] Connection reset by peer

though sometimes I get

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py", line 880, in request
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py", line 914, in _send_request
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py", line 719, in send
  File "<string>", line 1, in sendall
error: [Errno 32] Broken pipe

I'm fairly confident that my URLs are working (the GET handler is doing just fine, thank you). Logging indicates that the request is not actually making it to the handler code.

googling around brings me to issue trackers suggesting that the issue is in httplib's handling of an error while uploading a big file (this one is 3.7 mb).

So, I am not ashamed to admit that I am out of my depth here-- how can I determine what is causing the error? Am I formatting my request properly (p.s. I also tried b64encoding the body, with the same results)?. In a larger sense, is what I'm doing (to test, not in life) reasonable? Does it have anything to do with configurable settings on the dev server? Would these problems go away if I were to try putting this on Apache? Your help is very much appreciated.

Answer 1

It does appear that the issue is in the dev server's handling of large requests. After deploying to apache with mod_wsgi, this problem goes away. Still lots of open questions for me about RESTful file uploads...