Autotest performance slowdown

Question

I've been using ZenTest to run all the tests in my Rails project for years and it's always been quite nippy. However, on my Mac it has suddenly started taking 3 times as long to run all the tests. We have 1219 tests and for the past year it would run all the tests in around 300 seconds on average. Now though, it's taking almost 900 seconds:

Finished in 861.3578 seconds.

1219 tests, 8167 assertions, 0 failures, 0 errors
==============================================================================

I can't think of any reason why such a slowdown would occur. I've tried updating to the latest gem version, reducing the log output from the tests and regenerating the test database, all to no avail. Can anyone suggest a way to improve the performance?

Answer 1

When you have eliminated the impossible, whatever remains, however improbable, must be the explanation: if it's not the gem, not the database (did you check indexes ?), not your Mac, not Rails (did you upgrade recently), could it be the code ?

I'd check git/svn/cvs logs for the few most recent changes you made, and look for anything that might e.g. be slowing down queries.

If you can't find anything right away, profile the code to see where the time is going. This will be slower than just remembering something you did change (which almost always turns out to be the explanation in this kind of situation), but might point you in the right direction.

Performance issues can be frustrating because any number of factors can have an impact. A missing index on the DB. Network latency. Low memory conditions. Don't give up, keep Tilton's Law in mind.

Answer 2

You are really going to do a little bit more homework here, I doubt its ZenTest:

1. Grab a version of your code when stuff was great and dandy a few months ago. Run all the tests, output all the test durations to a spreadsheet or something.

2. Grab a current version of your code base and repeat the process in 1)

3. If durations are the same, something about your DB configuration or machine config has changed

4. If all the tests are slower on average this is a hard one to diagnose, but it would seem that there is a new bit of code thats running in each test.

5. If a handful of new tests are really slow, fix them.

Answer 3

So I finally solved this. Here's how in three easy steps:

1. Insert OSX Leopard CD
2. Completely reinstall Leopard from scratch
3. Reinstall Ruby, MySQL etc

After doing this the tests run in under 260 seconds.

I have no idea what happened but it certainly seems to have been a MySQL issue somewhere.