Server Costs for a Computing-Intense Application?

Question

Hi,

I have a scientific application that I built in Python (the application's 'critical areas' are optimized with Cython, for increased speed).

Every instance of the application is given a text file (with parameters) an an input. The application reads the parameters from the text file and, using data that is stored in the hard drive, runs and outputs the results of the calculations back into the hard drive. Every instance requires about 600MB of memory over the course of its operation.

Currently, I am running the app on my laptop (Intel Core 2 Duo, T7500, @2.2Ghz, 2GB RAM). Every 'instance run' on my laptop takes about 3 hours to complete.

Due to the needs of the project, there is a need for me to run 10000 instances. Obviously, it would take forever to do so on my laptop, hence the need for more computing power.

Knowing that every such an instance is independent from another, what would be the cost of a server that can run say 10 instances at the same time (I am on a budget...)? Can you recommend on a configuration?

Currently, I am using Windows XP but ideally, I would be happy to have the server installed with unix (ubuntu).

Answer 1

You can spin up Amazon EC2 standard instances (1.7GB / 1 slow core) for $0.085 per hour, or 23GB / 8core "cluster compute" instances for $1.60 an hour.

"One EC2 Compute Unit equals 1.0-1.2 GHz 2007 Xeon processor."

According to the tool, 10,000 "High-CPU Medium" instances with 5 EC2 Compute Units and 1.7GB each, for 3 hours, is $5100. This doesn't include the cost of getting your source data in and results out.

You can also bid on idle server time. The current "spot" price is about 1/3 the "on-demand" price, and fluctuates with demand. If you put in a low bid, your job may be interrupted by demand.

It is interesting to compare the cost of electricity for running your server/cooling with the cost of an Amazon instance. Commerical electricity rates are about 7.5¢/kWh here..

Answer 2

http://calculator.s3.amazonaws.com/calc5.html

It'd probably cost you about 15 dollars and you cut run it on a massive cluster of computers if the problem is easily parallelized.