Image Storage and CDN for websites

Question

Currently I am looking to move my websites images to a storage service. I have two websites developed in PHP and ASP.NET.

Using Amazon S3 service we can host all our images and videos to serve web pages. But there are some limitations using S3 service when we want to serve images.

1. If website needs different thumbnail images with different sizes from original image, it is tough. We have again need to subscribe for EC2 also. Though the data transfer from S3 to EC2 is free, it takes time for data transfer before processing image resize operation.
2. Uploading number of files in zip format and unzipping in S3 is not possible to reduce number of uploads.
3. Downloading multiple files from S3 is not possible in case if we want to shift to another provider.
4. Image names are case sensitive in S3. Which will not load images if image name does not match with request.

Among all these first one is very important thing since image resize is general requirement.

Which provider is best suitable to achieve my goal. Can I move to Google AppEngine only for the purpose of image hosting or is there any other vendor who can provide above services?

Answer 1

If you're using EC2, then S3 is your best option. The "best practice" is to simply pre-render the image in all sizes and upload with different names. I.e.:

/images/image_a123.large.jpg
/images/image_a123.med.jpg
/images/image_a123.thumb.jpg

This practice is in use by Digg, Twitter (once upon a time, maybe not with twimg...), and a host of other companies.

It may not be ideal, but it's the fastest and most simple way to do it. In terms of switching to another provider, you'll likely not do that because of the amount of work to transfer all of the files anyway. If you've got 1,000,000 images or 3,000,000 images, you've still got many megabytes of files.

Fortunately, S3 has an import/export service. You can send them an empty hard drive and they'll format it and download your data to it for a small fee.

In terms of your concern about case sensitivity, you won't find a provider that doesn't have case sensitivity. If your code is written properly, you'll normalize all names to uppercase or lowercase, or use some sort of base 64 ID system that takes care of case for you.

All in all, S3 is going to give you the best "bang for your buck", and it has CloudFront support if you want to speed it up. Not using S3 because of reasons 3 and 4 is nonsense, as they'll likely apply anywhere you go.

Answer 2

S3 is really slow and also not distributed. Cloudfront in comparison is also one of the slowest and most expensive CDNs you can get. The only advantage is that if you're using other AWS already you'll get one bill.

I blogged about different CDNs and ran some tests: http://till.klampaeckel.de/blog/archives/100-Shopping-for-a-CDN.html

As for the setup, I'd suggest something that uses origin-pull. So you host the images yourself and the CDN requests a copy of it the first time it's requested.

This would also mean you can use a script to "dynamically" generate the images because they'll be pulled only once or so. Just have to set appropriate cache headers. The images would then be cached until you purge the CDN's cache.

HTH