amazon-s3

S3 Amazon C# (SOAP vs REST)

If you were to create a professional application using S3 services, would you use their SOAP or REST interface (and why)? Assume using C# 3.5, VS2008, etc. (basically MS stack). Thank you ahead of time for your response(s). -Jessy Houle ...

Can't connect to AWS in Rails? What should I do to debug?

I have implemented paperclip in rails exactly as a variety of tutorials recommend. Upon trying to upload a photo I consistently get the following: AWS::S3::NoConnectionEstablished usr/local/lib/ruby/gems/1.8/gems/aws-s3-0.6.2/lib/aws/s3/connection.rb:213:in `connection' /usr/local/lib/ruby/gems/1.8/gems/aws-s3-0.6.2/lib/aws/s3/bas...

Web based client for Amazon S3

We are looking for a secure online solution to access our files stored on Amazon S3. We have about 3K files, mostly media and documents, that we need to make available to our employees on the move. We don't want to develop anything in-house if there is an existing solution. Please note that our employees are not technologically minded ...

Hadoop Input Files

Is there a difference between having say n files with 1 line each in the input folder and having 1 file with n lines in the input folder when running hadoop? If there are n files, does the "InputFormat" just see it all as 1 continuous file? ...

Amazon S3 client based on SOAP

I need a Win client for Amazon S3 that utilizes SOAP protocol for all transactions. As far as I see most solutions are REST based rather than SOAP. Any ideas? EDIT: Just want to clarify: please do not suggest using REST instead. I am perfectly aware of what can or cannot be done with either protocol. So if I ask for this specific solu...

Amazon S3 for Social Networks?

http://www.s3fm.com/ is really useful for uploading/viewing files for personal viewing. I was wondering if there was something good for social networks (using Amazon s3 or something similar), where it will only show the subset of files uploaded by a specific user (for example, limit access to the user's specific bucket). Can s3fm be a...

Browser Based Uploads to Amazon S3 ?

Amazon has instructions on how to write a form that posts directly to Amazon's S3 service via HTML POST. However, they are not very clear on how to create a simple form to upload directly to an S3 bucket, and never give an actual example html file... Is there a good example available where I can view the sample code? ...

Listing directories at a given level in Amazon S3

I am storing two million files in an amazon S3 bucket. There is a given root (l1) below, a list of directories under l1 and then each directory contains files. So my bucket will look something like the following l1/a1/file1-1.jpg l1/a1/file1-2.jpg l1/a1/... another 500 files l1/a2/file2-1.jpg l1/a2/file2-2.jpg l1/a2/... another 500 file...

How do I avoid a huge bandwidth bill if my website gets popular?

As a hobby, I'm writing a web application that will use some bandwith. Imagine I implement a feature that will produce custom 50 megabyte downloads for each interested user and maybe a megabyte or ten for uninterested visitors. I hope it will be popular but I don't want to have to take out a second mortgage on the house to pay for a mont...

is is possible to move/copy an s3 bucket to a different account?

Does Amazon provide a way to copy a bucket from one account to a different account? I am uploading several gb of files to my own bucket for a client app for development purposes, but when handing off the code I'm going to want to switch the bucket to their account (so I am no longer paying for the storage). Uploading is taking quite awh...

Paperclip, large file uploads, and AWS

So, I'm using Paperclip and AWS-S3, which is awesome. And it works great. Just one problem, though: I need to upload really large files. As in over 50 Megabytes. And so, nginx dies. So apparently Paperclip stores things to disk before going to S3? I found this really cool article, but it also seems to be going to disk first, and then do...

Force CloudFront distribution/file update

I'm using Amazon's CloudFront to serve static files of my web apps. Is there no way to tell a cloudfront distribution that it needs to refresh it's file or point out a single file that should be refreshed? Amazon recommend that you version your files like logo_1.gif, logo_2.gif and so on as a workaround for this problem but that seem...

What is involved with changing attachment_fu's storage scheme?

I have a rails application that is using attachment_fu. Currently, it is using :file_system for storage, but I want to change it to :s3, to allow for better scaling as more files get uploaded. What is involved with this? I imagine that if I just switch the code to use :s3, all the old links will be broken. Do I need to just copy the e...

PHP/Amazon S3: Query string authentication

I have been using the Zend_Service_Amazon_S3 library to access Amazon S3, but have been unable to find anything that correctly deals with generating the secure access URLs. My issue is that I have multiple objects stored in my bucket with an ACL permission of access to owner only. I need to be able to create URLs that allow timed access...

Listing buckets with AWS::S3 in Sinatra

I'm trying to access my Amazon S3 account using the aws-s3 gem, but no matter what I do, I can't even print a bucket list. Error: undefined method `bytesize' for #<AWS::S3::Bucket:0x1b9e488> From Code: # hello_world.rb require 'rubygems' require 'sinatra' require 'aws/s3' get '/' do connection = AWS::S3::Base.establish_connection...

How can I zip/tar files on S3 without first copying them to the server?

I have a Rails app that archives high-res images (and thumbs) on S3 (via the Paperclip plugin). I would love to allow users to download multiple high-res files with a single download. Is it possible to create a .zip/.tar file on S3 without first copying them back to the server? Our app runs on a large dedicated box and I'd love to save...

Amazon S3 Python Bulk File Transfer through Python

I want to tranfer files in around 1000 directories to an Amazon S3 bucket using Pythons S3 package. How could I do it ? ...

How to calculate the transfer bandwidth consumption of a file stored in Amazon S3?

I'm developing a file hosting service in ASP.Net where the users will have a certain amount of storage and transfer bandwidth according to the plan they purchased. Considering the fact that I will be using Amazon S3 to store the files, I'm looking for a way to track the amount of bandwitdth used by each file, so that I can associate thi...

s3 cloudfront listing all contents in bucket to anonymous users

Hi Anonymous Users can browse to our public distributions or s3 folders and view a list (xml) of all files. We have set read permissions. Can anyone recommend how to prevent users from viewing the list of files in the bucket, while still allowing them to access and download individual files Thanks Dom ...

CloudFront - How to upload images

I have 3 questions related to Amazon CloudFront. How do I upload images? Can I use FTP? Does Amazon allow me to rsynch a local directory to CloudFront so that I have a replica directory and hierarchy on CloudFront? Thanks in advance ...