I need more VMs, should I expand my current server, or just get another server to host them?

Question

I've currently got in my office a few physical servers hosting a number of VMware guest OSs off VMWare server.

We need to expand the number of VMs we need to use to do some testing and development in house.

A smattering of them are world accessible for our clients, however they are not doing anything particularly load intensive (CPU, bandwidth or memory)

One of the servers has a second CPU socket and some extra space for memory. We are currently struggling between the decision to:

1. Upgrade the server. Currently it's a 1xQuad core Xeon with 4 gigs of ram.
   • Simple upgrade would be to add a 2nd identical CPU to make it a 2xQuad core Xeon
   • as well as upgrade to 8 gigs of ram
2. Get another server of identical specs (1x quadcore 4gig) and put the new VMs on there.

Each VM instance doesn't utilize more than 1 virtual CPU or more than a gig of ram tops. Most of them are rather lightweight linux machines for specific tasks.

The issue we're wondering about is how much of a hit is there to VMWare and its guest OS's, if we completly trick out our server with the CPUs and RAM and pile on as many VMs as possible?

This seems like the VMware prescribed route for saving power and shelf space, especially for our low utilization...but is there any experience that suggests that more physical machines are still better?

The upgrade route for the machine is still an attractive option for us because it is about 1/3 to 1/4 the price of getting a comparably priced new machine.

Answer 1

I'd say that upgrading would probably be sufficient. But heck, I don't know what your ultimate requirements are.

If you're doing anything where you're going to need fail over, then you'll want the second machine for redundancy.

That said, it may be worth your time to upgrade from VMWare server to VMWare esx or the more competitively priced VMWare esxi (free). esx(i) have almost no footprint on your hard drive (32MB) or in memory. Assuming your hosting on Windows, that's a big savings. If you've already got a stripped down linux os, the change may not be worth it.

Answer 2

Building up the current machine is a savings in purchase price, rack space, power draw, and cooling. In another 18 months when you want to expand again, a new computer will do all the current one can do + all the new load you want to put on it, and be cheaper than you expected.

Buying a second host now is a good way to provide redundancy, in case of a hardware issue. Assuming you have good backups of your VMs, you can recover from a hardware failure pretty quickly.

It depends on what you really need.

Answer 3

Don't forget to consider disk space. Unless you have a SAN, it can be tough to keep up with direct attached storage requirements (space and speed) of a handful of VMs. When you've got 6-8 machines all using the same disk controller, it can get a bit contentious.

With that said, I almost always look to expand a virtual server before going out to purchase a new physical server. ascalonx and Jay Bazuzi make good points though, having somewhat redundant virtual host can be essential to keeping downtime brief if the first server ever goes down. I always like to know that I have another virtual server that can take the load of any failed VM host that might go down. If you don't have that, you may want to consider getting a bit more hardware.

Answer 4

Another option is Amazon EC2. This can allow you a lot of scalability in terms of how many machines you can have running at one time. And the cost is pretty low for what you get.

Answer 5

We have a similar configuration as to what you'd be looking to upgrade to.

• Dual Quad Core Xeon
• 8Gb RAM (server can handle up to 32Gb)
• VMWare ESXi

We're running about 25 VMs at the moment and will eventually end up with the full 32Gb of RAM as our demands increase. Most are just testing machines for our support team to replicate customer environments so are just idling 24/7, we've got 8Gb physical ram with 16Gb allocated to VMs - usually not a problem unless we're actively using multiple Vista or Server 2008 VMs.

The first upgrade we went thru was to get a hardware RAID card (to support ESXi) which boosted performance like you wouldn't believe (software raid + vista = ugh!), next steps will be upgrading ram to 12Gb, then 16Gb as we add more and more VMs. Once memory price falls enough we'll start replacing pairs of 2Gb sticks with pairs of 4Gb ones.

Even under heavy load, the bottleneck on our machine is memory. So I'd suggest upgrading your current machine, there's plenty of life left in it yet.