How to create structured automatic testing with selenium (junit or testng)? What is your best practice for this?

Question

I have written tests that are using selenium for functional testing. I'm using Java. I would like to be able to create structure between tests. Is it possible to do it with jUnit or TestNG?

Would like to be able to create test structure that will produce report like this:

Top test FAIL
- Module1 test PASS
-- Module1Class1 test PASS
-- Module1Class2 test PASS
--- Module1Class3Method1 test PASS
--- Module1Class3Method2 test PASS
- Module2 test FAIL
-- Module2Class1 test FAIL
--- Module2Class1Method1 test PASS
--- Module2Class1Method2 test FAIL
--- Module2Class1Method3 test PASS

Answer 1

You cannot, because this is not the format that Selenium interprets tests. Selenium supports several different language syntaxes, the easiest of which are JS and HTML table rows. You could use the record feature of the Selenium IDE from Firefox if you do not want to hand write your tests to fit a certain language syntax.

Answer 2

did you consider using Selenium Remote Control? http://seleniumhq.org/projects/remote-control/

Answer 3

I would recommend looking at the JUnit or TestNG XML reports. You should be able to transform them with a bit of XSLT to provide a new HTML report with your required format.

Answer 4

I am not complete sure if i am right here..but i think this can help you: http://www.jamesnetherton.com/blog/2007/07/02/Creating-a-Selenium-test-suite/

You can group your tests with it in an very easy structered way.

Answer 5

The best pattern I have seen for organizing the code behind selenium tests is the page object pattern:

http://blog.m.artins.net/acceptance-tests-with-jbehave-selenium-page-objects/

Here's a Java helper library:

http://code.google.com/p/webdriver/wiki/PageFactory

Answer 6

I just have found the solution for this question so I'm putting here link so that others can benefit from it. beust.com I haven't try that yet, though.

Update: After trying it I can generate results like this:

Module1.Class3.Method1 PASS
Module1.Class3.Method2 PASS
Module2.Class1.Method1 PASS
Module2.Class1.Method2 FAIL
Module2.Class1.Method3 PASS

The fail method name is "Method2" it is located in class "Class1" and it is package "Module2".

I have used all the standard possibilities of TestNG (a lot of it) + I have overwritten the TestListenerAdapter using ITestResult methods: getName() and getTestClass().getName()

It's not really the structure I was looking for, but little bit parsing can tell me where the fail was. And as a plus I don't have to name methods with class and package name in it.

Answer 7

In TestNG you can define suites via the configuration file (testng.xml) which should match your needs. You could structure it to have a test group for every Module. A failure in a test group renders the whole test as a failure. But i think you could also do that in JUnit, which i am not too familiar with.

What is nice in TestNG is, that you can define dependencies. These will enforce a certain logical order for test execution and will skip tests that depend on failing tests, instead of letting them fail as well. Makes analysis a lot more easier and tests end earlier because all those tests that are potentially doomed to fail will be left aside.

But like i said earlier, i think you can do that in JUnit as well. It's more a matter of taste. And if you decide otherwise, it is not a big undertaking to convert from JUnit to TestNG or visa versa.