How best to resize images off-server

Question

I have a site that users upload images to. Those images are resized to various dimensions. I currently do that on my server, but was thinking of processing that on AWS or something similar. Is that a good idea?

Answer 1

You could use an EC2 server to do the resizing and store them in S3 if you anticipate the need for a large scale out and load. Ultimately, It would would depend on alot of factors depending on your growth and budget. What is the current load on your server and are you near capacity? What is the size and amount of CPU it takes to resize the images and can they be cached and reused? How are the images used?

Answer 2

You need to figure out how much it takes to process your images and how many you anticipate processing.

If your server can process them in real time with your peak load, don't worry about it right now.

I'd say that if your server can handle the load, but not in real time, I'd build a queue on the server and have a cron job or daemon process the images outside of the request. If the queue starts growing or your server starts getting overloaded, you can move it to another machine.

If that other machine can't handle the load, you should be able to, somewhat easily, add more machines to process the queue.

At that point, you're probably pretty sophisticated and you can start taking advantage of EC2 for things like scaling on demand.

Answer 3

I've been using EC2 and dedicated servers for similar tasks, and here are some tips:

1. Use as little IO as possible. If you intend on using things like ImageMagick, know that writing to disk is a waste of io resources (unless you want to cache it anyway.) PHP/GD lets you send jpeg directly in http response.

2. Free RAM as soon as possible: release source and resized images whenever you can. Eliminate the chance of your server swapping out memory.

3. EC2 is extremely slow with context switches. So try as little as possible. Processors like Gimp are -extremely- slow (a scale down that took 0.2 seconds on dedicated, took 7 seconds on EC2.) Also, ImageMagick is extremely slow, period. If you can, use PHP/GD for all of your processing.

4. Remember to set maximum memory. You will need two copies of the image in memory (source and scaled down.)

5. Last but not least: start with real time. Don't start with overkill solution of mysql queues and external daemons. These will prove hard to maintain in the long run. So give real-time a try before you build a rocket.

Good luck!