Integration Testing an Entire *Existing* Application (w/ automatic execution of test suite)

Question

Hi there,

I have just joined a team working on an existing Java web app. I have been tasked with creating an automated integration test suite that should run when developers commit to our continuous integration server (TeamCity), which automatically deploys to our staging server - so really the tests will be run against our staging web app server.

I have read a lot of stuff about automated integration testing with frameworks like Watir, Selenium and RWebSpec. I have created tests in all of these and while I prefer Watir, I am open to anything.

The thing that hasn't become clear to me is how to create an entire test suite for an application, and how to have that suite execute in it's entirety upon execution of some script. I can happily create individual tests of varying complexity, but there is a gap in my knowledge about how to tie everything together into something useful.

Does anyone have any advice on how to create a full test suite and have it execute automatically?

Thanks!

Answer 1

To run a Watir test that is in a file, just run the file:

$ ruby tests_1.rb

To execute tests in several files run all files. You can create a file that will run them all (for example all_tests.rb):

load "tests_1.rb"
load "tests_2.rb"

and then just run the file:

$ ruby all_tests.rb

I am not familiar with TeamCity, but you should be able to just run all_tests.rb from it.

Answer 2

I ended up writing the tests using Rspec to make assertions against Watir (Celerity, to be precise) objects. This allowed me to use Rake to automate the running of the tests. There is a few good articles out there about using Rspec and Rake together. Our build server (teamcity) has hooks for Rake tasks so this works well. It took me a while to piece everything together in my mind so I thought I would post the eventual solution here.

Answer 3

Typically you are going to use Rake to automate the test execution. Assuming you are using Test::Unit for your testing you would setup your Rakefile With the following contents:

require 'rubygems'
require 'rake'
require 'rake/testtask'

Rake::TestTask.new do |t|
  t.libs << "test"
  t.test_files = FileList.new 'test/**/ts_*.rb'.sort
  t.loader = :rake
  t.verbose = true
end

This configures all test suite files under your project "test" folder by default. You can then run them with the command below:

rake test

and it will then execute all your test suites for your entire project. You can tell it to run a specific test by using the following syntax:

rake TEST=path/under/test/folder/tc_filename.rb test

Since you are using TeamCity you can then create a build and use the Rake runner to execute your test suites. TeamCity will pull all of the test information (output, stack traces, etc.) into the UI just like it does with JUnit. It is a very good integration.

For reference, your test suites would look something like this:

require 'test/unit'
require 'path/relative/to/your/tests/tc_some_test1.rb'
require 'path/relative/to/your/tests/tc_some_test2.rb'

This way you can sequence your test cases within each test suite as desired.