views:

216

answers:

3

Background
The QA department where I work has a lot of automated blackbox tests that interact with our applications via the GUI and the command line. Currently, the automated tests output their results to standard out where we then manually enter the final pass/fail result into a spreadsheet.

We would prefer to have a system where the automated test automatically saves detailed test results to a file. We would then have a web page that the testers and developers could access to view the detailed test results and any necessary attachments. It would generate reports of the test results by project and version number.

Question:
What system would you recommend for test report generation? We need a system where our tests will automatically be inserted into new reports and that is preferably open source. I'm interested in what your company actually uses or what you have found useful in managing test results.

Our QA department is capable of building a simplified version of this system from scratch, however we would prefer not to reinvent this.

+1  A: 

reStructuredText is a very happy medium between writing to stdout and formal documentation. There are several scripts to convert from rst to other formats such as html.

You could mostly keep the system you have in place -- you'd only have to add a couple "tags" around the text, but unlike HTML tags, these are more readable characters. In fact, it's very close to the markdown you use when asking/answering here on StackOverflow.

The stdout text remains overall very readable by humans, but then it's as simple as adding one script in the chain to render to HTML or PDF for instance.

This page has a very good example of what it looks like in plain text and rendered forms.

Mark Rushakoff
A: 

Maven has an an extensive site mechanism, it does require you to bend to its will though, so that might rule it out for you.

Once configured you get a standard set of reports generated on each build, that can be packaged as a jar if you wish, or deployed directly to your build results site. There are plugins for many of the major reporting tools, such as Cobertura/Emma, Junit, JDepend etc.

The maven-site-plugin publishes its own sites if you want to have a look.

Rich Seller
+1  A: 

We are now using Testopia. it is integrated with Bugzilla, it is nice to have everything at the ame place. It uses the same XMLRPC API interface as Bugzilla.

Julien