views:

64

answers:

2

We all know that App Engine limits you to 1 MB for most input/output requests. But with the recent BlobStore API, you are allowed to upload large files in full by POSTing to a dynamically generated URL.

According to the sample, here is what the HTML form would look like:

self.response.out.write('<html><body>')
self.response.out.write('<form action="%s" method="POST" 
  enctype="multipart/form-data">' % upload_url)
  self.response.out.write("""Upload File: 
<input type="file" name="file"><br> 
<input type="submit" name="submit" value="Submit"> 
</form></body></html>""")

But how can we do this asynchronously using JavaScript techniques introduced with HTML5? This is a snippet of what I have so far:

xhr.open("POST", post_url); // the post URL given by App Engine
xhr.overrideMimeType('text/plain; charset=x-user-defined-binary');
xhr.setRequestHeader('Cache-Control', 'no-cache');
xhr.setRequestHeader('X-File-Name', file.fileName);

// After loading the binary data (last time we only read as base64 string)
// Tell xhr to start the upload
myBinaryDataReader.addEventListener("loadend", function(evt){
   xhr.sendAsBinary(evt.target.result);
}, false);

// Initiate the binary reading on the file, when finished it will 
// upload asynchronously 
myBinaryDataReader.readAsBinaryString(file);

You'll notice that this technique sends the raw binary file as the POST body. Which is fine, it works without needing the BlobStore for up to 1 MB. In Python, to read the file, I just use:

img_data = self.request.body # got my image data now

However, with BlobStore, I'm supposed to use

upload_files = self.get_uploads('file')  # 'file' is file upload field in the form

But I'm not using an HTML form with input type=file, I'm using an XmlHttpRequest -- how can I make App Engine "think" it is a file from an HTML form, and thus "grab" the file data?

My code, unmodified, results in an error

File "C:\Python26\lib\cgi.py", line 583, in keys
    raise TypeError, "not indexable"
TypeError: not indexable
+1  A: 

You may want to check out my blog posts on uploading to the blobstore (1, 2, 3), as well as this recent cookbook post.

Nick Johnson
thank you that shed a lot of light on this. It looks like I might be S. out of luck because the part that makes this work only works in firefox. That's because only Firefox supports reading the file directly, and to do the multi-part encoding, which unfortunately App Engine requires at this time for the BlobStore. looks like I have to switch to amazon s3.
Michael Butler
PLUpload, which I use in the later parts of my article, supports several upload methods, and any modern browser ought to support at least one. Failing all else, regular form upload with a dynamically generated URL, as in the cookbook post, would work fine.
Nick Johnson
Michael Butler
You can use multipart encoding in an XmlHTTPRequest - you'll just have to create the multipart encoding yourself.
Nick Johnson
indeed you can Nick, but if you intent to encode a dropped file with multipart encoding, you can only do it in Firefox right now. This is because Chrome and Safari do not have access to read binary file data (they can send a raw file as POST body though).
Michael Butler
A: 

The general consensus is that, so far, App Engine's BlobStore upload API will only accept multipart encoded POST data... in other words, an HTML input type=file form. Or, you can use Firefox to read a user's binary file (via drag and drop) with the FileReader API and reconstruct the multipart encoding manually. Then you can submit the data asynchronously with XmlHttpRequest.

As of writing, Chrome and Safari do not support the FileReader object so they cannot break apart binary files and thus not send multipart encoding asynchronously.

Note that the drag N drop + XmlHttpRequest method of uploading files to App Engine still works in all 3 aforementioned browsers if under 1 MB.

Michael Butler