tags:

views:

55

answers:

1
Q: 

Best way to manage an S3 Bucket?

I have been trying for a month and a half now to transfer files and update ACL permissions.

My bucket in question is 28gigs, and has over 45,000 objects in it.

It generally takes three days to do it. I've tried the following :

  • Bucket Explorer
  • CloudBerry
  • s3sync
  • s3cmd
  • s3 organizer
  • s3fox

And quite frankly, none of them can handle it. All of them die after a few hours of running the process of copying these files or updating their ACL's.

Does anyone know what I can do?

A: 

Sorry for stating the obvious here but why not update them in multiple sessions?

mark  2010-08-26 19:25:59
What do you mean exactly?
Trip  2010-08-28 04:44:41
Just queueing so many transfers while reliant on a network connection and Amazon's S3 API is likely to be problematic. Surely uploading then updating the permissions a couple of gigs at a time would make more sense.
mark  2010-08-28 10:41:19
Right..Not a bad idea. I guess I never assumed it would be this tedious. ;D
Trip  2010-08-28 19:06:26

related questions

Is there a rake task for backing up the data in your database?
How to represent cross-model information in MVC?
WYSIWYG editor gem for Rails?
Best Solution For Authentication in Ruby on Rails
Acts-as-readable Rails plugin Issue
Associating source and search keywords with account creation
Any tips on getting Rails to run with an Access back-end?
Being as DRY as possible in a Ruby on Rails App
How Do You Secure database.yml?
What IDE to use for developing in Ruby on Rails on windows?
Is Ruby On Rails ready for the Enterprise?
OpenID authentication in Ruby on Rails
Why Doesn't My Cron Job Work Properly?
Why does sqlite3-ruby-1.2.2 not work on OS X?
Ruby mixins and calling super methods
Why all the Active Record hate?
Haml: how do I set a dymanic class value?
Learning Ruby on Rails any good for Grails?
How to sell Python to a client/boss/person with lots of cash
How do I create a new Ruby on Rails application using MySQL instead of SQLite?
Ruby On Rails with Windows Vista - Best Setup?
What is good forum software to add to an existing Rails application?
How do I fix 'Unprocessed view path found' error with ExceptionNotifier plugin in rails 2.1?
Frequent SystemExit in Ruby when making HTTP calls
Implementation of "Remember me" in a Rails application.