I want to make a copy of the folders and images on my s3 bucket for my development server. How can I do that?
+1
A:
Hey,
If you are using linux, and just want to drag copies down to the local filesystem, then you could use s3sync:
If you wanted to access the files directly on s3, you could mount s3 as a fuse filesystem locally, but beware that accessing files using this method is dependent on your connection, so there could be speed issues. I've used s3fs perfectly well for accessing backups etc:
(can only post one link atm, but google s3fs - it's hosted on googlecode)
If you just need a copy, then s3sync is the easiest option.
Hope this helps.
scourou
2010-08-04 20:59:59
Yes, actually, sorry, I meant that how do i duplicate a folder on the s3 from the s3. So if I have a folder named production. And I want a carbon copy of that data for my testing server. How do I duplicate that? I found CloudBerry works well, but windows is the herpes of operating systems, and I'd rather find a mac alternative. Know of any?
Trip
2010-08-05 13:26:06
Sorry - don't know of any mac alternatives. I guess you could run a cron on OSX to sync down from 'production' to a local folder using s3sync, and then push that up to the 'testing' bucket. Something like this should do it (presuming you have s3sync set up ok):"s3sync.rb -r productionbucket: /local/folder"To get them down, then "s3sync.rb -r /local/folder testingbucket:"To put up to testing
scourou
2010-08-05 16:56:20
Interesting. I found CloudBerry for PC works, its just excruciatingly slow. I assume the same goes for S3FS. I have 20 gigs of images, roughly over 20,000 images. Ran the process the three and a half days straight, and not sure if it made a dent.
Trip
2010-08-09 15:42:08
I think to a certain extent, you'll be restricted by what your connection can handle. Have you tried syncing them down, then syncing them back up to a separate bucket? Maybe try on a subsection of the data to get a comparison between speeds for cloudberry and s3sync.I'd leave s3fs alone for now, it's not entirely appropriate to your needs - i.e. you can use it to do what you need to, but s3sync is more likely to help quicker.
scourou
2010-08-10 10:55:23
Hey just downloaded something called Bucket Explorer, and it works awesome. It works on all OS's, and it seems it handles cloning, duplications, copying and moving with a really easy to understand API.
Trip
2010-08-10 13:07:59
A:
I have to say, in conclusion, I recommend using a GUI. They've already laid out the work for you.
My best recommendation is Bucket Explorer ( works on all OS's)
Second runner up is CloudBerry ( only on PC's )
Bucket Explorer has a sweet very easy to understand GUI, and has a lot of great perks, analytics, and usability that outweighs all the others I experimented with.
Trip
2010-08-11 12:55:38