So I know this is a common question but there just doesn't seem to be any good answers for it. I have a bucket with gobs (I have no clue how many) number of files in them. They are all within 2k a piece. 1) How do I figure out how many of these files I have WITHOUT listing them? I've used the s3cmd.rb, aws/s3, and jets3t stuff and t...
Hi I am hoping someone can help me to understand why I am getting varied results with some PHP code I have written to upload files to S3 then call an EC2 instance to perform actions on the uploaded file. Here is the order i do things - 1) use S3 class to put file $result = s3 -> putObjectFile($uploadDIR, $bucket, $name, S3::ACL) ...
I've seen ways to upload files directly to S3 and I've seen ways to upload a file already on the server. I need to modify the file's name before sending it on to S3, but I don't want to store it locally and then upload it, if I can help it. Is there a way to put it in a buffer or something? Thx. ...
Hi, Is it possible to upload files to amazon S3, using their servers (i.e. my server shouldn't be used in the upload process)? I also want the URL to appear to be uploading to my server, is this possible? ...
Can you put permissions on files that are uploaded to S3? Or if someone knows the guild/url to the file they can access it publicly? If yes, what kind of permissions can we set? ...
Hi, I need to retrieve a result from a mysql database using a user defined function that recreates the following PHP code - $signature = urlencode(base64_encode((hash_hmac("sha1", utf8_encode($string_to_sign), awsSecretKey, TRUE)))); Are UDF's up to the job? What are the security implications for storing the AWS secret key in the U...
What do you see as the advantages and disadvantages of Amazon Web Services S3 compared with Google Application Engine? The cost per gigabyte for the two is, at the time I ask, roughly similar; I have not seen any widespread complaints about the quality of service; so I think the decision of which one to use may depend on the API (of all...
I'm putting assets to S3 with expiry headers, which successfully get passed through to CloudFront distributions. However, requests after the expiry don't result in CloudFront requesting a fresh file from the origin server. I am also finding that query string version tokens don't force a fresh asset, and I don't want to use full, file na...
So, I have a dedicated server. I host about dozen or so small sites. Is there a real benefit in using S3(or Mosso) for my image and static file hosting? My server has more than enough disk space, or am I completely missing the point of S3? I keep reading about how wonderful and cheap it is, and I ask myself "self, why aren't you using ...
I am a keen follower in news related to Amazon S3. It seems to be reliable, scalable and a cheap service. However, web hosts do not seem to be overly scared. I have not noticed any major drops in prices since Amazon and Google released their 'Cloud' based platforms. Is there a reason for this? Or are web developers simply not aware of ...
I'm using CNAMEs together with S3/CloudFront to serve some static files like js, css, images, etc. I do it to make the bucket's URL pretty and because I think is better have all targeting to my site and in case in some future I want to move those files the change should be transparent. Today reading blogs I saw some think CNAMEs are e...
I have a fairly large amount of data (~30G, split into ~100 files) I'd like to transfer between S3 and EC2: when I fire up the EC2 instances I'd like to copy the data from S3 to EC2 local disks as quickly as I can, and when I'm done processing I'd like to copy the results back to S3. I'm looking for a tool that'll do a fast / parallel c...
Trying to understand S3...How do you limit access to a file you upload to S3? For example, from a web application, each user has files they can upload, but how do you limit access so only that user has access to that file? It seems like the query string authentication requires an expiration date and that won't work for me, is there ano...
I am currently developing a rails application that tries to copy/move videos from one bucket to another in s3. However i keep getting a proxy error 502 on my rails application. In the mongrel log it says "failed to allocate memory." Once this error occurs the application dies and we must restart is. ...
I've got files on Amazon's S3. They are named with a unique ID so there are no duplicates. I am accessing them using an authorized URL. I need to be able to pass them through to the browser, but I need to rename them. Right now I'm using fopen, but it is downloading the file to my server before serving the file to the browser. How can I ...
I'm creating an application that will be hosted on amazon EC2 and a lot of the data that'll be saved is more document oriented (as well as saving tweets and such related to those documents). Right now I'm at a crossroads... should I use simpleDB or couchDB? Whats the pros/cons of using either? Should I just try both for a month and dec...
I have been looking at the usage reports from Amazons S3 service and noticed that there is a DataTransfer-out-bytes charge for GetObject operations (ok i understand this one) and also a DataTransfer-out-bytes charge for HeadBucket operations. What is HeadBucket, when is this request made? cheers ...
I have some files stored at amazon. all in private mode, and since I need provide users a way to download these files, each time an user needs to download a file I just create a authenticated url according to Authenticating REST Requests and the user can download the file for a gap of 5 minutes. BUT once the url is generated I can see i...
I'm looking to for a service that is similar to Amazon S3, a simple service to store and retrieve arbitrary data (and meta-data), but one that runs locally in your own data center. Strictly speaking, I'm not sure whether you would call this a CDN or a lightweight CMS. It must be horizontally scalable (both for storage and bandwidth) a...
I have a large collection of data chunks sized 1kB (in the order of several hundred million), and need a way to store and query these data chunks. The data chunks are added, but never deleted or updated. Our service is deployed on the S3, EC2 platform. I know Amazon SimpleDB exists, but I want a solution that is platform agnostic (in c...