views:

137

answers:

1

So I'm struggling with this for a second day in a row and still nothing. Found few solutions on the internet but still I'm getting "Internal Server Error" when trying to send files with POST. The idea is as follows : I'm sending a file opened in python's shell to a django function on my server that will read and store the file there. I've tried urllib, urllib2 (with and without poster module), httplib and Multipart methods. Here's the whole list of what I've tried together with results :

#Variables
In [35]: url = "http://www.address"
In [36]: values = {'key1': 'value1', 'key2': 'value2', 'key3': 'value3'}
In [37]: dataurllib = urllib.urlencode(values)
In [39]: dataposterlib, dataheaders = multipart_encode(values)
In [40]: dataposterlib
Out[40]: <generator object yielder at 0x15a0410>
In [41]: headers
{'Content-Length': 491,
 'Content-Type': 'multipart/form-data; boundary=13936c2ddba0441eab9c3f4133a4009e'}
In [43]: f = open("winter.jpg", "rb")
In [45]: filegen, fileheaders = multipart_encode({"file": f})
In [46]: filegen
Out[46]: <generator object yielder at 0xc5b3c0>
In [47]: fileheaders
Out[47]:
{'Content-Length': 39876,
 'Content-Type': 'multipart/form-data; boundary=0aa7a1b68d714440be06e166080925ec'}


#Working:

1. urllib.urlopen("http://www.address", data=urllib.urlencode({"key": "value"}))
2. urllib.urlopen(url, data=urllib.urlencode({"key": "value"}))
3. urllib.urlopen(url, data=urlib.urlencode(values))
4. urllib.urlopen(url, urlib.urlencode(values))
5. urllib.urlopen(url, data=dataurllib)
6. urllib.urlopen(url, dataurllib)
7. urllib.urlopen(url, data=urllib.urlencode({"file": f})) : works, but doesnt send file

8.  import cookielib
    from utils.multipart import MultipartPostHandler
    cookies = cookielib.CookieJar()
    opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cookies), MultipartPostHandler)
    params = { "username": "bob", "password": "riviera", "file": f }
    opener.open(url, params)

#Not working: 

4. u = urllib.urlopen(url, "string")
5. u = urllib.urlopen(url, data="string")
7. u = urllib2.urlopen(url, data=urllib.urlencode(values)) : Error 500 but sends data
6. u = urllib2.urlopen("http://www.fandrive.yum.pl/event/upload/", data=urllib.urlencode({"key": "value"})) : Error 500 but sends data
7. u = urllib2.urlopen(url, data=urllib.urlencode({"key": "value"})) : Error 500 but sends data
8. u = urllib2.urlopen(url, data=dataurllib) : Error 500 but sends data
9. u = urllib2.urlopen(url, dataurllib) : Error 500 but sends data
10. In [31]: req = urllib2.Request(url, f, fileheaders)
    urllib2.urlopen(req)                                : Error 500
11. req = urllib2.Request(url, filegen, fileheaders)
    urllib2.urlopen(req)                                : Error 500, very seldomly sends data
12. req = urllib2.Request(url, dataurllib)
    urllib2.urlopen(req)                                : Error 500, sends data

I've found somebody with the same problem so I borrowed his django function (which is based on this post :

def upload(request):
    logging.debug("GO")
    for key, file in request.FILES.items():
        path = '/path/'+ file.name
        logging.debug(path)
        dest = open(path.encode('utf-8'), 'wb+')
        logging.debug(dest)
        if file.multiple_chunks:
            loggin.debug("big")
            for c in file.chunks():
                dest.write(c)
        else:
            logging.dobug("small")
            dest.write(file.read())
        dest.close()
    destination = path + 'test.txt'
    logging.debug(destination)
    file = open(destination, "a")
    file.write("file \n")
    file.close        
    return 0

I've noticed that even when my function runs, the files sent are not saved, as well as the code after destination... is skipped. Just as there would be timeout before it reached this place. Wireshark shows POST request being sent, but server always gives 500 error. How is this happening ? I've tried few combinations with my 2 servers as well as sending from local computer but still nothing. What am I missing here ? I'm using the same code as everybody else on the internet. Even built-in urllib2 module always gives error even though urllib works.

EDIT

I've changed server to another one and finally I have some solid error log (receiver side):

[Sat Oct 16 20:40:46 2010] [error] [client IP] mod_python (pid=24773, interpreter='host', phase='PythonHandler', handler='django.core.handlers.modpython'): Application error
[Sat Oct 16 20:40:46 2010] [error] [client IP] ServerName: 'xxx'
[Sat Oct 16 20:40:46 2010] [error] [client IP] DocumentRoot: 'path'
[Sat Oct 16 20:40:46 2010] [error] [client IP] URI: '/error/internalServerError.html'
[Sat Oct 16 20:40:46 2010] [error] [client IP] Location: None
[Sat Oct 16 20:40:46 2010] [error] [client IP] Directory: None
[Sat Oct 16 20:40:46 2010] [error] [client IP] Filename: 'path/error/internalServerError.html'
[Sat Oct 16 20:40:46 2010] [error] [client IP] PathInfo: ''
[Sat Oct 16 20:40:46 2010] [error] [client IP] Traceback (most recent call last):
[Sat Oct 16 20:40:46 2010] [error] [client IP]   File "/usr/lib/python2.5/site-packages/mod_python/importer.py", line 1537, in HandlerDispatch\n    default=default_handler, arg=req, silent=hlist.silent)
[Sat Oct 16 20:40:46 2010] [error] [client IP]   File "/usr/lib/python2.5/site-packages/mod_python/importer.py", line 1229, in _process_target\n    result = _execute_target(config, req, object, arg)
[Sat Oct 16 20:40:46 2010] [error] [client IP]   File "/usr/lib/python2.5/site-packages/mod_python/importer.py", line 1128, in _execute_target\n    result = object(arg)
[Sat Oct 16 20:40:46 2010] [error] [client IP]   File "/usr/lib/python2.5/site-packages/django/core/handlers/modpython.py", line 228, in handler\n    return ModPythonHandler()(req)
[Sat Oct 16 20:40:46 2010] [error] [client IP]   File "/usr/lib/python2.5/site-packages/django/core/handlers/modpython.py", line 220, in __call__\n    req.write(chunk)
[Sat Oct 16 20:40:46 2010] [error] [client IP] IOError: Write failed, client closed connection.
[Sat Oct 16 20:40:46 2010] [error] [client IP] python_handler: Dispatch() returned non-integer.
+2  A: 

MAJOR EDIT:

Im very sorry, I gave you the wrong code. The working code, which I use, is based on this: http://code.activestate.com/recipes/146306-http-client-to-post-using-multipartform-data/ so I won't repeat it here and claim It being mine.

Because of your comment I read RFC1867 and realised I gave you the wrong code (I use this, too, it is convenient, if you control both sides).

Your Errorlog shows IOError: Write failed, client closed connection., which I suppose means that the server either tries to write some data (the file?) somewhere it doesn't should or I where it has no file system permissions to write. You should check where this is and look for write permission for the user the server runs as.

knitti
I was already trying with httplib previously :/ How can I process this error ? It just shows me sample code, that everybody get when receiving 500 error. And server log shows "Premature end of script headers:" but not exactly at the times when I was sending my requests so I'm not sure if it's in any way connected.
owca
look at the error log at the receiving side
knitti
i get nothing on the receivers side. damn I've just totally wasted 2 days without any results.
owca
if the receiving side throws a HTTP 500 and doesn't log anything, you have severe problems to fix.
knitti
but what if i'm getting it when sending from my localhost, should I look for the source on my server?
owca
The HTTP 500 comes from the server to which you POST. If this sits on your localhost, look there. It is however irrelevat, from _where_ you POST.
knitti
Just checked your solution on another server, and the data is sent (I currently don't have privileges to check if the file was saved properly), returns 200 status but I've checked it in wireshark and this request looks completely different than when I upload some image on let's say imageshack. Imageshack gives me : "POST / HTTP/1.1 (GIF89a) and within the request I have MIME part, whilst sending from python : "application/form-data..." and no MIME part.
owca