views:

2187

answers:

5

Hi all,

I am trying to port my first Django 1.0.2 application to run on OSX/Leopard with Apache + mod_python 3.3.1 + python 2.6.1 (all running in 64-bit mode) and I am experiencing an occasional error when uploading a file that was not present when testing with the Django development server.

The code for the upload is similar to what described in the Django documentation:

class UploadFileForm(forms.Form):
    file  = forms.FileField()
    description = forms.CharField(max_length=100)
    notifygroup = forms.BooleanField(label='Notify Group?', required=False)

def upload_file(request, date, meetingid ):
    print date, meetingid
    if request.method == 'POST':
        print 'before reloading the form...'
        form = UploadFileForm(request.POST, request.FILES)
    print 'after reloading the form'
        if form.is_valid():
            try: 
                handle_uploaded_file(request.FILES['file'], request.REQUEST['date'], request.REQUEST['description'], form.cleaned_data['notifygroup'], meetingid )
            except:
                return render_to_response('uploaded.html', { 'message': 'Error! File not uploaded!' })
            return HttpResponseRedirect('/myapp/uploaded/')
    else:
        form = UploadFileForm()
    return render_to_response('upload.html', {'form': form, 'date':date, 'meetingid':meetingid})

This code normally works correctly, but sometimes (say, once every 10 uploads) and after a fairly long waiting time, it fails with the following error:

IOError at /myapp/upload/2009-01-03/1
Client read error (Timeout?)
Request Method: POST
Request URL:    http://192.168.0.164/myapp/upload/2009-01-03/1
Exception Type: IOError
Exception Value:    
Client read error (Timeout?)
Exception Location: /Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/site-packages/django/http/multipartparser.py in read, line 406
Python Executable:  /usr/sbin/httpd
Python Version: 2.6.1
Python Path:    ['/djangoapps/myapp/', '/djangoapps/', '/Library/Frameworks/Python64.framework/Versions/2.6/lib/python26.zip', '/Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6', '/Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/plat-darwin', '/Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/plat-mac', '/Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/plat-mac/lib-scriptpackages', '/Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/lib-tk', '/Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/lib-old', '/Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/lib-dynload', '/Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/site-packages']
Server time:    Sun, 4 Jan 2009 22:42:04 +0100

Environment:

Request Method: POST
Request URL: http://192.168.0.164/myapp/upload/2009-01-03/1
Django Version: 1.0.2 final
Python Version: 2.6.1
Installed Applications:
['django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.sites',
 'django.contrib.admin',
 'myapp.application1']
Installed Middleware:
('django.middleware.common.CommonMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware')

Traceback:
File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/site-packages/django/core/handlers/base.py" in get_response
  86.                 response = callback(request, *callback_args, **callback_kwargs)
File "/djangoapps/myapp/../myapp/application1/views.py" in upload_file
  137.         form = UploadFileForm(request.POST, request.FILES)
File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/site-packages/django/core/handlers/modpython.py" in _get_post
  113.             self._load_post_and_files()
File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/site-packages/django/core/handlers/modpython.py" in _load_post_and_files
  87.                 self._post, self._files = self.parse_file_upload(self.META, self._req)
File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/site-packages/django/http/__init__.py" in parse_file_upload
  124.         return parser.parse()
File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/site-packages/django/http/multipartparser.py" in parse
  134.             for item_type, meta_data, field_stream in Parser(stream, self._boundary):
File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/site-packages/django/http/multipartparser.py" in __iter__
  607.         for sub_stream in boundarystream:
File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/site-packages/django/http/multipartparser.py" in next
  421.             return LazyStream(BoundaryIter(self._stream, self._boundary))
File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/site-packages/django/http/multipartparser.py" in __init__
  447.         unused_char = self._stream.read(1)
File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/site-packages/django/http/multipartparser.py" in read
  300.         out = ''.join(parts())
File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/site-packages/django/http/multipartparser.py" in parts
  293.                 chunk = self.next()
File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/site-packages/django/http/multipartparser.py" in next
  315.             output = self._producer.next()
File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/site-packages/django/http/multipartparser.py" in next
  376.             data = self.flo.read(self.chunk_size)
File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/site-packages/django/http/multipartparser.py" in read
  406.         return self._file.read(num_bytes)

Exception Type: IOError at /myapp/upload/2009-01-03/1
Exception Value: Client read error (Timeout?)

I tried to run everything using mod_wsgi and no difference.

Does anybody know what am I doing wrong?

Thanks in advance for your help!

ppdo

=====

Updated:

Though I succeeded uploading large files (60+ MB), when it fails it fails with no evident relationship with the size of the upload, i.e. it fails also with 10kB files that have successfully been uploaded before.

A: 

How large is the file? It may take long enough to upload that the upload script times out, so try increasing the execution time for that script.

Dana the Sane
Sorry, that's not it: I can upload large files and it still fails with small ones. See updated question. Thanks anyway.
A: 

Though I succeeded uploading large files (60+ MB), when it fails it fails with no evident relationship with the size of the upload, i.e. it fails also with 10kB files that have successfully been uploaded before.

This is not an answer. Please update your question with additional information.
S.Lott
A: 

I would chase down the exception value

Client read error (Timeout?)

this seems odd enough. Try reading this thread

Bartosz Radaczyński
+4  A: 

Using mod_wsgi made the problem go away for Firefox.

Limiting my research to an interaction problem between Apache and Safari, I stumbled upon this bug report for Apache https://bugs.webkit.org/show_bug.cgi?id=5760 that describes something very similar to what is happening and it is apparently still open. Reading this gave me the idea to try and disable the keepalive and, though I need to test it more extensively, it seems the problem is gone.

A simple:

BrowserMatch "Safari" nokeepalive

in the Apache configuration did the trick.

A: 

Long time unanswered here, but having experienced this myself and tried a few things to resolve it. It seems in my case, it happened if someone canceled a download (or lost connection) during an upload.

You can try this yourself to see if this is what is causing the "IOError: request data read error" for you: upload a large enough file that you can unplug your ethernet cable from your computer or router. Wait a bit and see if you get the error. I do everytime!

Bryan