I'm looking for a way to upload a file to s3. I am using django. I am currently using amazon's python library for uploading along with the following code: View: def submitpicture(request): fuser = request.session["login"] copied_data = request.POST.copy() copied_data.update(request.FILES) content_type = copied_data['file'].get('conte...
I'm researching this for a project and I'm wondering what other people are doing to prevent stale CSS and JavaScript files from being served with each new release. I don't want to append a timestamp or something similar which may prevent caching on every request. I'm working with the Spring 2.5 MVC framework and I'm already using the g...
mysqldump is probably not the best idea on running backups every x hours but is the one that we are currently using. The backups are around 150 Megs each so sending it to other machines could waste many gigs of bandwidth. Is there any tool or application or script that could automate the creation of the backup and the upload to s3 am...
Hi, If I had a million images, would it be better to store them in some folder/sub-folder hierarchy or just dump them all straight into a bucket (without any folders)? Would dumping all the images into a hierarchy-less bucket slow down LIST operations? Is there a significant overhead in creating folders and sub folders on the fly and ...
What types of Technologies does Amazon.com use Internally? Databases? Programing languages? Web technology? What technology is used to build ec2? s3? simpleDB? ...
I'm looking for details of the cloud services popping up (eg. Amazon/Azure) and am wondering if they would be suitable for my app. My application basically has a single table database which is about 500GB. It grows by 3-5 GB/Day. I need to extract text data from it, about 1 million rows at a time, filtering on about 5 columns. This ext...
Background: I'm designing the authentication scheme for a REST web service. This doesn't "really" need to be secure (it's more of a personal project) but I want to make it as secure as possible as an exercise/learning experience. I don't want to use SSL since I don't want the hassle and, mostly, the expense of setting it up. These SO q...
I have a private bucket that stores full sized images for clients, all thumbnails and smaller sizes are on the webserver. When the user has multiple images they would like to download, I'd like to zip the aforementioned images, then deliver them as one file to the user. Currently, the only way I can think of this happening is by transf...
Relating to this question, http://stackoverflow.com/questions/117810/upload-files-directly-to-amazon-s3-from-asp-net-application, is there any way to do this and have a progress bar? ---- EDIT ---- Two days later and still no luck with a direct way. Found one thing that looks promising but not free: http://www.flajaxian.com/ Uses fla...
Okay this is a fairly broad question. This is my first App and I'm not sure the best way to go about this. The app is on the IPHONE. I have a 'Restaurant' class. The restaurant has many different attributes and opening times. I currently store a restaurant in an instance of nsdata (it complies to NSCoding) LOcal storage is easy and I j...
The web service that I want to run on AWS has to store and retrieve user data, present it to the user via a website, and needs to be able to parse the sitemaps of a few thousand sites every 10 min or so. Which components of AWS, such as S3, EC2, and CloudFront do I need to use. A short synopsis about the purpose of each component would b...
Would one be violating the the terms of agreement with Google App Engine if one were to use Amazon's S3 service with Google App Engine? I do know there is restrictions on what you can and can't do but I was not sure of this. Having these two combined features, one could provide one heck of a system at a relatively low cost compared to h...
I am aware that Amazon S3 is usable in Flex via an AIR application ( http://stackoverflow.com/questions/186768/can-you-use-amazon-s3-via-flex ) but was curious if the same applied to a browser ( swf ) compilation of Flex source. At http://code.google.com/p/as3awss3lib/ it states that: "It only works in Apollo because of restrictions in...
Say, you have a Rails (or actually any other kind) application running on Amazon EC2, and whenever you perform some maintenance, which might include shutting down or re-creating your instances, you might want S3 to display your "maintenance mode" page instead of your app. So, basically, that might mean: Uploading text saying when you'...
I'm trying to retrieve HTTP HEAD information from a S3 resource using Python. Unfortunately the response always returns a 403 error. I am using code that was suggested here but unfortunately this does not work for S3 requests. Here is the code (host and key details omitted): >>> import httplib >>> conn = httplib.HTTPConnection("www.myb...
I have large data files stored in S3 that I need to analyze. Each batch consists of ~50 files, each of which can be analyzed independently. I'd like to setup parallel downloads of the S3 data into the EC2 instance, and setup triggers that start the analysis process on each file that downloads. Are there any libraries that handle an as...
While the post @ http://highscalability.com/amazon-architecture explains Amazon's architecture in general, I am interested in knowing how Amazon S3 is implemented. Some of my guesses are A distributed file system like HDFS http://hadoop.apache.org/core/docs/current/hdfs_design.html A non relational persistent DB like CouchDB http://co...
How can I change the Amazon S3 url from http://bucket.amazons3.com/imagepath.jpg to http://image.mydomain.com/imagepath.jpg EDIT: Basically the whole reason for this is to hide the amazon s3 url from my users. I was thinking about a HttpModule that would redirect the request from image.mydomain.com to bucket.amazons3.com. But ...
When getting a URL for something in an Amazon S3 bucket it can append a signature to the end to confirm that the user has permission to view the object and the URL looks like so: https://mybucket.amazonaws.com/mykey?AWSAccessKeyId=myaccesskey& Expires=1235241261&Signature=t5vFBWXaN0DvVaWfck9n2%2fmTzOU%3d These URLs were comin...
I'm using Amazon S3 to host images. The S3 bucket is private, so I generate a temporary URL (using Right AWS) with a 5-minute expiry to allow the image to be rendered. The URL looks like this (note: URL below will not work): https://mybucket.s3.amazonaws.com:443/attachments%2F30%2Fsmall.png?Signature=J%2BXzQd95myCNv0Re8arMhuTFSvk%3D&...