Easy acceptance testing with specification

Question

I look for a tool/framework to make automatic acceptance-testing. The interface to create new tests should be so easy, that a non-programmer (customer, boss) will be able to add specifications for which will be tested automatically.

It should be some way to execute the tests from command-line, to include a run of the tests in automatic builds.

I prefer Java and Open-Source, but my question isn't restricted in that way.

What do you can recommend and please explain why your tool/framework is the best in the world.

Answer 1

http://fitnesse.org/ appears to meet all of the qualifications you want. It is one I have used with success.

Answer 2

What you ask for appears to be for a very well-defined system with a very specific sets of inputs and a high degree of automation built-into the system or developed for your system.

Commercial applications such as HP Quick Test Pro isn't non-technical enough and requires an additional framework such as one from Sonnet, which is a step in the right direction, but neither is open source or java-based.

Even with a framework in place, it's quite a bit of work to make this work in an automated way. I'd like you to consider the time needed to develop the framework vs the time to manually run these tests and verify that you are using your time well.

Answer 3

I've found a framework named Concordion that may fulfill my needs.

Answer 4

Another framework you may want to look at is Robot Framework. To see how test cases look like, take a look at the Quick Start Guide.

Answer 5

How about Cucumber:

Feature: Acceptance testing framework

  Scenario: an example speaks volumes
    Given a text example
    When it is read
    Then the simplicity will be appreciated

You would need a developer to discuss with the boss what each of those lines really means and implement the step definition to drive it:

Given /^a text example$/ do
  file.open("example.txt", "w") { |file| file.write "text example" }
end

When /^it is read$/ do
  SystemUnderTest.read("example.txt")
end

Then /^the simplicity will be appreciated$/ do
  SystemUnderTest.simplicity.should be_appreciated
end