How do you handle cross-browser compatibility testing?

Question

Working for a small web design/development firm, we're sometimes tasked, by clients, who are unwilling to budge on their browser compatability list. Any given project could need to be tested against some combination of IE 6, 7 & 8, Firefox 2 & 3, Opera, Safari & Chrome running on OS X, Windows XP, Vista or Linux.

Since the bulk of development can (and should) be done in a modern, standards compliant browser, I had considered a headless Mac Mini running OS X and a few VMs that would take remote connections. Unfortunately, the Mini maxes out at 2GB which limits the number of simultaneous VMs. The iMac, with it's 20" screen seems overkill for a machine that'll spend much of its time idling in the corner. Having a single Macbook is

Having every developer multi-boot or load their machine down with VMs and then deal with hacks to run multiple versions of the same browser just becomes a hassle to keep running, so having a central system seems like it'd be good way to have everything set up & known to be working.

How has everyone else dealt with this in the past? Since there's plenty of Macs around the building, having the box only handle the Windows & Linux browsers is an option - even if it wouldn't be near as cool.

Answer 1

For the Windows testing, setup a headless, powerful PC with lots of RAM and disk space to run Windows VMs. Then let your coworkers remote desktop to it when they need to test those. You can run Vista x64 (to support all the RAM) as the host and test IE7 directly, then use Virtual PC (free) and the MS-provided VMs for testing IE6 & 7 on XP. I think there's an IE8 for XP and Vista image too. On each image you can also install Chrome, Firefox and Opera or just test them on the host OS, since there's really no difference for those between OSs. The key is a lot of RAM (like 8GB) so that each VM can run smoothly.

Answer 2

There are some services out there, such as Litmus and browsershots.

I test the site myself (I'm usually the sole developer), and sometimes get others to check it out too. I'm running a mac, so I've got vmware fusion with XP to test those browsers.

I generally draw up a table on a piece of paper, each browser gets a column. Any issues are written in the far left column and I mark any browsers that show that issue on its row. Once I've tested, I'll then go through the issues and mark each place it rears its head complete when resolved. It's pretty primitive and takes time, but I'm not dealing with massive projects, so it doesn't hurt.

Answer 3

Just use it - http://browsershots.org/

Answer 4

Well, if you have a lot of money, you could ofcourse buy 10 different computers, and installed a different browser on each of them. But its a pain in the ass to develop on one computer while test on an other computer...

So the best solution (imho) is to develope on OS X, and then test IE6 and IE7 via parrallels/VMWare. I save a lot of time and enegy by doing all my development on one computer, and witout the need to reboot!

Answer 5

I do development on OSX and test via different parallels images. I do a lot of javascript/interactive work, so vnc into a vm isn't quite sufficient for that sort of work. However, it's quite adequate for just your standard html/css type development.

Another alternative to using Vista x64 is to setup numerous different VMWARE instances on a linux server. Another alternative, you could setup a number of windows VM's on Amazon's cloud, if you don't want to maintain a physical machine.

As for the mac browsers, just switch. All the cool kids are doin' it. C'mon.

Answer 6

Unfortunately I have to support IE6, & 7 as well. I used an app called multiple IEs to install them both on the same machine. Somebody really needs to come up with standards to prevent this kind of thing.

Answer 7

Yet another option would be BrowserSeal. Since it is an application and not a web service, it is much faster compared to pretty much everything else.