views:

439

answers:

1

I've managed to get email attachments onto Amazon S3 from a GAE incoming email, but does anyone know a technique to get an attachment, like an image, into the blobstore.

Any help would be much appreciated.

Code so far (with help from Alex)

upload_url = blobstore.create_upload_url('/upload')
msg = MIMEMultipart()
msg.set_type('multipart/form-data')
msg.set_payload({'file': content})
result = urlfetch.fetch(upload_url, payload=urllib.urlencode(msg), method=urlfetch.POST, headers={'Content-Type': 'multipart/form-data'})
+3  A: 

To receive mail in your GAE app, follow the docs here: in particular, you'll get a instance of class InboundEmailMessage with an attachments attribute which, and I quote:

is a list of file attachments, possibly empty. Each value in the list is a tuple of two elements: the filename and the file contents.

Then, per these GAE docs, you "create an upload URL" and in your upload handler (typically a subclass of BlobstoreUploadHandler) you use get_upload to get BlobInfo instances and put their metadata somewhere that will later let you fetch them back as your app may require.

Finally, you POST the data (that you have from attachments, above) to your own freshly generated "upload URL", e.g. using urlfetch.fetch (with method-POST and a payload in standard application/x-www-form-urlencoded encoding for the "form" that the user would be filling in if they were uploading the data directly, which is the "normal" way to put data in the blobstore -- e.g. you can use urllib.urlencode to prepare the payload).

That "self-POST" will be using another instance of your app to "receive" the data into the blobstore (while the instance that received the email waits, as fetch is synchronous).

Alex Martelli
Wouldn't you need a multipart encoded form request (as thee's a file attachment there) ?I've been trying to code something that'll post to blobstore from server code for some time now without success....
Eran Kampf
Yes, I do believe you need multipart (and shd have mentioned that the `email` module can help preparing that).
Alex Martelli
@Tomás, I don't have working code because I don't have blobstore in my GAE account (as I haven't enabled billing there). For the encoding, having used `email`, use `msg.flatten()`. I suggest you edit your Q to show this code so you can format it.
Alex Martelli
I've tried a whole manner of things now Alex, to no avail. I don't know enough about what to encode, how to encode it and when. Are you sure that these procedure is even possible?
Tomás Flannery
@Tomás, I'm sure it's possible, I'm not sure on all details -- I'll try to find the time to develop a full solution but it's hard without actual blobstore access, as I mentioned.
Alex Martelli
Happy to add you to my list of developers, feel free to upload any solution you like, still very much in the testing stage.
Tomás Flannery
Added you to Developer list Alex, would really appreciate any demonstration of Image Attachment to Blobstore, think it would really benefit the GAE community. Feel free to erase any solution currently live.
Tomás Flannery
@Tomás, I'm working on a simple variant of the "blobstore helloworld" at http://code.google.com/appengine/docs/python/blobstore/overview.html which also allows you to fake the upload with a post, trying to leverage http://code.activestate.com/recipes/146306-http-client-to-post-using-multipartform-data/ -- just so you know what I'm up to;-)
Alex Martelli
Sounds perfect, good luck and I look forward to seeing it.
Tomás Flannery
I've got a simplistic prototype up at http://www.aleax.it/blosto.zip but I can't try it on my local SDK, since in the SDK **any** "post to the same app" just times out (I've verified that with a trivial post that easily works to any other app). That's quite a bother as it makes it unfeasible to try on the local SDK any app that "posts to itself" (even though it would work once uploaded since a new instance of the app is involved in that case:-(. But you might want to look at how the multipart post body's constructed -- that part should work.
Alex Martelli
Alex, thanks for posting that code. I tried your approach and it worked great for attachments that are under 1 MB. However, urlfetch is failing when processing attachments > 1 MB. This is problematic because what I'm trying to do is process emailed photos and shrink them down to a size I can store in the datastore. I'm able to do this without a problem using traditional html forms / blobstore, but when trying your code for emailed attachements I get "call urlfetch.Fetch() was too large." Any ideas?
ryan
@ryan, unfortunately I have no idea of how to work around this limitation (which is not in the blobstore but in urlfetch). I guess it needs a new feature request on App Engine's tracker (to let the app eng team know about the need for attachment->blobstore "fast path" data flow).
Alex Martelli