My company runs a site where we allow users to upload files. I'm worried about scaling as the traffic and client volume increases.
I don't believe that we'll have an all-at-once traffic hit, but it will gradually increase over time. But over time it may get rather large, and not something I want to manage.
If anyone has experience with it I'd appreciate the input.
REF: Amazon S3
The first data point you'll probably come across is that the service is currently being used by some really big players such as Twitter. The second data point you'll come across is that there was recently a high profile outage of their service which caused lots of their clients to go down: http://sitening.com/blog/lights-out-for-amazon-simple-storage-service-s3-shutters-twitter-and-thousands-of-other-websites/
Whether this was a one time occurrence, and whether this is an acceptable risk for your company is something only you can decide :-)
I've only used it through 3rd parties, like backup-manager on Linux, but I know that ITConversations uses S3 for storage and delivery, with good results. The charges (for a commercial site) are very reasonable, and I think it's a stable and workable solution. I'm considering using it for MP3 storage and delivery myself.
Take a look at Don MacAskill's blog. He's the CEO of SmugMug. They have several TB of data on S3. He writes in great detail about their use of S3 and EC2.
http://blogs.smugmug.com/don/category/amazon/
His article S3 outage - We weren’t affected talks about how their architecture robustly handles temporary unavailability of S3.
There are lots of other people writing about running a portion of their business using S3 as well. The common approach seems to be to design your application to handle the possibility of S3 going away for awhile, then syncing up from your local data center's resources up to S3 when it comes back.
In summary, S3 can be a valuable resource in your business application's architecture. You just can't treat it like local storage.
Since some "big players" have already jumped on board, I wouldn't be too worried.
I believe that at least these 2 big "apps" are using S3 without issue:
Pownce (from Kevin Rose, of Digg.com) http://pownce.com/
Dropbox (real cool file sync/sharing) http://www.getdropbox.com/
I've been using S3 for almost a year and I've had no problems. Really hard to find any other "hosting" more reliable.
My software uploads 100-400 small files per client per day. No problem so far.
I know that services such as 37Signals BaseCamp product use it, as well as JungleDisk's backup system. I've been using both and have had NO issues ever.
However, I believe the first item on this list here hits on a key point, if you are 100% business critical that it must be available 24X7X365 you will want to most likely plan for a "what if" scenario.
its worth it, Plenty of apis available, and it doesn't cost much.
as for downtime which they got a really bad rap for, everything is going to go down, no one can guarantee uptime, but amazon are going to do a better job at keeping everything running than most.,