How to be successful in web user interface testing ?

Question

We are setting up a Selenium test campaign on a big web application. The first thing we've done was to build a framework which initialize SQL data in database before the test, launch the test, archive results and then clear data.

We've integrate that in a Maven 2 process, run every day by TeamCity on a dedicated database.

We've set up several Selenium tests now but It's not as used as planned.

Reasons are that tests are sometimes broken for other reasons than regressions (data may have changed, stored procedure may have been recompiled and so on).

I would like to know if there are big success in user interface testing and if so, reasons to that. Commons errors may also interest me.

Answer 1

I use http-unit which has the added benefit of working before any styling has been added to the page.

http://httpunit.sourceforge.net/ You can attach the tests to run in the integration test phase for maven2.

From the site

Written in Java, HttpUnit emulates the relevant portions of browser behavior, including form submission, JavaScript, basic http authentication, cookies and automatic page redirection, and allows Java test code to examine returned pages either as text, an XML DOM, or containers of forms, tables, and links.

Answer 2

If you want reliable unit tests, you need to have the same input. Starting state of the database is the input. So, you need to have the same starting database each time. Of course, if you wish to do testing with different input, you need to create another unit test (as results will obviously not be the same).

When I do stuff like this, I always use the same database as a starting point. Of course, some of the tests might fail without modifying the database is correct way, so some other subsequent tests might fail as well even though they wouldn't otherwise. If your unit-test tool allows, you should define dependencies between tests to make sure that those tests will not be run at all when the 'parent' one fails.

Answer 3

Testability helps a lot. The biggest win for testability in web apps is if all of the HTML elements you need to interact with on the page have unique and consistent attributes. If the attributes you are using to identify the HTML elements (Selenium uses xpath) are not consistent/reliable from build-to-build, or session-to-session, your test scripts will fail. Also, these attributes must be unique, so that the automation tool (in this case Selenium) can reliably find the object on the web page.