Test framework using spreadsheet for Selenium RC / JUnit / Java

Question

We have a smoke test that we run every morning to check a number of applications which involves logging in, executing a simple operation and logging out.

The test at the moment is a collection of Selenium IDE scripts which were imported into Selenium RC as Java and run inside JUnit running inside Netbeans.

What we would like to do is run the test from a spreadsheet i.e each line of the spreadsheet has the application URL, the login parameters, some titles and text to check and the log out sequence.

At the moment, our simple POC simply has one JUnit Test class of the form:

@Test public void testTestSmokeCheck() throws Exception { ... }

This calls a class which loops through the spreadsheet and does:

selenium sel = new DefaultSelenium
sel.start
sel.open
...
sel.close

for each line of the spreadsheet.

This works but the problem is that many lines of the spreadsheet are compressed into one JUnit test which either all passes or all fails.

What we would like is for each line of the spreadsheet to be a separate JUnit test.

This way each line of the spreadsheet would result in either red or green which would be far more meaningful.

Any ideas how to achieve that?

Answer 1

Not exactly a spreadsheet, but you might want to look into FitNesse. It drives tests from tables on Wiki pages, and prints out red/green pass/fail.

You can do multiple pages and test suites, which should solve your problem.

Answer 2

You can use the Parameterized runner (the JavaDoc has an example). If a test fails, the failure message will include the index of the data, which could correspond to the row in the spreadsheet.

If the number of rows isn't huge, I recommend writing a test case per row and getting rid of the spreadsheet. Sooner or later you will want to do specific logic for some of the pages, and a spreadsheet will no longer model that well.