Re-scaling automatic build from 2-cpu pizza-boxes to multi-cpu monster(s) ?

Question

Our current automated build consists of 1 master box and 4 core-2-duo pizza boxed (at around 2.5Ghz), with 3 GB memory each, all runing ubuntu linux. (Using bamboo)

I have been asked to recalibrate this for running on one or more solaris boxes. Our current build consists of a mix of java builds, long running integration test builds and automated selenim tests - a fairly typical large project build. Our current environment has sufficient capacity, and we could probably lose a core or so without problem.

We are looking at 8 CPU solaris machines with "large" amounts of memory (16-32GB? I can choose). Or maybe several 4 cpu machines ?

Can I expect behaviour to scale in a fairly linear manner within a single box? I am looking for actual experiences here; "theoretical" observations about how IO may become a problem are not very interesting.

Answer 1

Actually I find that builds spend most of their time doing IO of some sort. So increasing the number of CPUs doesn't help as much as you would think. However, additional machines will have additional disks and network bandwidth. So they may help more. However, you can improve the configuration of a single box and possibly get the same performance.

How many builds you you typically have running at once? If you typically have 2 builds queued now (and 2 running) then having four builds running at once is an improvement. But more than that isn't going to make much difference.

Eight core machines with 64 GB of memory are surprisingly standard for a new server these days. In this configuration, I would suggest getting an SSD drive to improve disk access times and running four to eight builders/agents and that could be enough for if you typically have eight builds waiting at any time or less.

Answer 2

If your tests are long running, you may benefit from having two builds, a quick build which runs all the fast tests, and a slower full build which might be only run over night.