What do you use to Unit-Test your Web UI?

Question

The company I'm currently working for is using Selenium for Uniting-Testing our User Interface. What do you use to Unit-Test your Web UI and how effective do you find it?

Answer 1

We use Selenium Core, but are switching gradually to Selenium RC which is much nicer and easier to manage. We have written lots of custom code to make the tests run on our Continuous Integration servers, some of them in parallel suites to run faster.

One thing you'll find is that Selenium seems to restart the browser for each test (you can set it not to do this, but we got memory problems when we did that). This can be slow in Firefox, but is not too bad in IE (one time I'm thankful for Bill Gates's OS integraion).

Answer 2

We are using QuickTestPro. So far it is effective, but the browser selection is limited. The nicest part is the ability to record your browser's activity, and convert it into a scriptable set of steps. There is also a nice .Net addin so if you have any validation code you need to do for the different stages of your test, you can write methods in an assembly and call them from your script.

Answer 3

Well, if you've designed your application properly, you won't have scads of logic inside the UI anyway. It makes much more sense to separate the actual work getting done into units separate from the UI, and then test those.

If you do that, then the only code in the UI will be code that invokes the backend, so simply testing the backend is sufficient.

I have used NUnit ASP in the past (at my job), and if you insist on unit testing your UI, I would strongly advise you to use ANYTHING but NUnit ASP. It's a pain to work with, and tests tend to be invalidated (needing to be revised) after even the most minor UI changes (even if the subjects of the tests don't actually change).

Answer 4

We have been using JSunit for a while to do unit tests... it may not be the same kinds of tests you are talking about, but it is great for ensuring your JavaScript works as you expect.

You run it in the browser, and it can be set in an Ant build to be automatically run against a bunch of browsers on a bunch of platforms remotely (so you can ensure your code is cross-browser as well as ensure the logic is correct).

I don't think it replaces Selenium, but it complements it well.

Answer 5

We use Watin at my place of employment, we are a .net shop so this solution made a lot of sense. We actually started with Watir (the original ruby implementation) and switched after. It's been a pretty good solution for us so far

Answer 6

I've used WATIR, which is pretty good. I liked it because it's Ruby and allows for testing interactivity, available elements and source code parsing. I haven't used it for a while but I assume it's gotten better.

It's supposedly being ported to Firefox and Safari, but that's been happening for a while now.

Answer 7

Check out Canoo Web Test. It is open source and built on the ANT framework.

I spent some time working with it for a graduate course on Software QA and it seems to be a pretty powerful testing tool.

Answer 8

Selenium Grid can run your web tests across multiple machines in parallel, which can speed up the web testing process

Answer 9

I mostly use CubicTest, which is an eclipse plugin that lets you define tests graphically. It can export/run tests through several libraries, including watir and selenium. Most people just use the Selenium runner though.

Full disclosure: I'm one of the developers, so I'm kind of biased :)

Take a closer look here: cubictest.openqa.org

-Erlend

Answer 10

Selenium is for Integration testing, not Unit testing. It's a subtle, but important difference. The usage I usually see is for sanity checking a build. i.e., have a test that logs in, a test that (for example) submits a story, makes a comment, etc.

The idea is that you're testing to see if the whole system is working together before deployment, rather than have a user discover that your site is broken.

Answer 11

I'm a huge fan of Selenium. Saying 'unit-testing your web ui' isn't exactly accurate as some of the comments have mentioned. However, I do find Selenium to be incredibly useful for performing those sort of acceptance and sanity tests on the UI.

A good way to get started is using Selenium IDE as part of your development. Ie, just have the IDE open as you're developing and write your test as you go to cut down on your dev time. (Instead of having to manually go through the UI to get to the point where you can test whatever you're working on, just hit a button and Selenium IDE will take care of that for you. It's a terrific time-saver!)

Most of my major use case scenarios have Selenium RC tests to back them up. You can't really think of them as unit-tests along the lines of an xUnit framework, but they are tests targetted to very specific functionality. They're quick to write (especially if you implement common methods for things like logging in or setting up your test cases), quick to run, and provide a very tight feedback loop. In those senses Selenium RC tests are very similar to unit-tests.

I think, like anything else, if you put the effort into properly learning a test tool (eg, Selenium), your effort will pay off in spades. You mention that your company already uses Selenium to do UI testing. This is great. Work with it. If you find Selenium hard to use, or confusing, stick with it. The learning curve really isn't all that steep once you learn the API a little bit.

If I'm working on a web app, its rare for me to write a significant amount of code without Selenium RC tests to back it up. That's how effective I find Selenium. :) (Hopefully that'll answer your question..)

Answer 12

We use Visual Studio 2008 Tester Edition.

Pros: Very good at capturing user interaction

Captures Ajax calls

It is very easy to map user input to a database, XML or CSV file

The captured test can be converted to C# for more control

The same tests can be used for load testing and code coverage

Cons:

VS2008 Tester Edition is a seperate SKU from the normal Developer Edition, which means extra cost

You may be alergic to Microsoft ;-)

We have used it very effectively on projects, however there a lot of effort involved in keeping tests up to date, every time you change a screen the test may need to be re-recorded

We tend to keep the tests short and sharp, do one thing and get out instead of recording 10 minutes worth of clicking around in a single test.

We have a few standard UI test types:

Menu Test: Log in as a specific user (or user type/role) and make sure all the required menu items are available

Validation Test: Open a page and click save without entering any data, ensure that all the validation warnings appear. Complete required fields one at a time and check that the warning messages disappear when they are supposed to.

Search Test: Search using data from your database or a data file and ensure the correct data is returned by the search

Data Entry Test: Create new recrords from a data file, cleanup the database to allow tests to run multiple times

UI Testing is quite time consuming but the comfort feeling you get when a few hundred tests pass before you release a new version is priceless.

Answer 13

We use WatiN for system testing, and QUnit for JavaScript unit testing.

Answer 14

Molybdenum is built over Selenium and has some additional features.