Identifying slow-running tests

Question

In ruby's test/unit, the software indicates how long it takes for the tests to run, and the series of passes, errors and fails act like a pseudo-progress bar.

Apart from using code profiling tools or running tests individually, is there an easy way of telling which test methods are fast and which ones are slow?

Answer 1

When I need to do this in a large test suite, I override Test::Unit::TestCase setup and teardown. It doesn't give precise measurements, but it can help assess relative speed.

module Test
  module Unit
    def setup
      @start_time = Time.now
    end

    def teardown
      puts "#{@method_name}: #{Time.now - @start_time}s"
    end
  end
end

Answer 2

According to Sarah's answer, I would prefer another solution:

Setup like Sarah wrote, but the Teardown should add the test name and the execution time in a list.
So you can evaluate this list, for sorting or whatever. I don't know Ruby, so I don't know if it would work.

Here is some Java code for JUnit to explain my thoughts...

public class ExecutionTimeTest {

  public static ArrayList<Double> executionTimes;

  public double start;

  @BeforeClass
  public static void initializeList() {
    executionTimes = new ArrayList<Double>();
  }

  @AfterClass
  public static void printExecutionTimes() {
    int i = 1;
    for (Double time : executionTimes) {
      System.out.println("Test " + (i++) + ": " + time);
    }
  }

  @Before
  public void startExecutionTime() {
    start = System.currentTimeMillis();
  }

  @After
  public void calculateExecutionTime() {
    executionTimes.add(System.currentTimeMillis() - start);
  }
}

Answer 3

If you're using the test-unit gem, running the tests in verbose mode will give you information on how long each test took:

ruby test/test_unit.rb --verbose
Loaded suite test/test_unit
Started
ChaserTestCase: 
  test_handle_funny_characters_in_class_method_names:   .: (0.000645)
  test_handle_funny_characters_in_instance_method_names:.: (0.000523)
  test_modify_and_unmodify_class_method:                .: (0.000534)
...
Finished in 0.019894 seconds.

MiniTest will also have benchmarking.