Transfering 100,000 images to S3. What's the quickest way?

Question

I'm looking for the fastest way to transfer ~100,000 images from a server to my Amazon S3 bucket.

Would that be to loop through every image and do a separate PUT request? I was hoping there was a quicker way, but haven't found anything.

Answer 1

Not familiar with S3, but if there is no bulk upload tool, you can maybe speed things up by looping over your images in multiple parallel processes, until you max out your network connection.

Answer 2

You can try to use Image Uploader batch upload control. Or if this is a one-time task I consider using some Amazon sample.

Anyway, if you want to use PUT/POST request for uploading, one file per request - is Amazon's API restriction and it seems to me there's nothing to be done.

Answer 3

Maybe try the snail mail?

Amazon Import/Export

Answer 4

If you are going to try to upload them, you might want to act soon to save some money: http://aws.amazon.com/s3/#pricing:

Data Transfer

• $0.030 per GB – all data transfer in April 1, 2009 through June 30, 2009 – 3rd Anniversary Celebration! (Note: data transfer in will return to its normal price of $0.100 per GB on July 1)