What is a good Code to Test Ratio?

Question

I am using RSpec for writing tests.

What do you think, is a good Code to Test Ratio?

Answer 1

A good test to code ratio is one that allows you to feel confident in the code you have written and also allows you to refactor with confidence you will know what you area ffecting in the rest of your application. I have had test ratios ranging from 1:1.5 to 1:2.5 or so, it can really vary depending the complexity of your application.

Answer 2

Code to Test Ratio is a bit of a misleading statistic. A much better method is to use rcov, you can easily use it by running rake spec:rcov

Whilst 100% code coverage is sometimes held up as an absolute target, you quickly run into the law of diminishing returns. Personally I aim for 90% code coverage in all production code; even though this is mainly arbitrary I find it much easier to have a target number to aim for.

Answer 3

It varies. Simple code I'd expect as much test code as production code. Complicated code can easily deserve twice as much test code. Do Test Driven Development and won't have to worry about the ratio since everything in the code was driven by a test and that is what is important.

Answer 4

We're talking about opinions at the moment. A good code-to-test ratio is one where your code is covered to the extent that it needs to be to allow both confidence in what is written and confidence that, when you are refactoring, you will know what is breaking all around you.

Numbers are good, but putting too much stock in them can be just as dangerous.

Answer 5

My goal is no untested code revealed by rcov and heckle. When you get all the coverage you can get with rcov, then you can run the code through heckle. Heckle modifies your code and shows you which modifications were not caught by tests.

rspec knows about heckle. After installing the heckle gem: "spec foo_spec.rb -H Foo". Oh, and if you get a bizarre error, install version 1.2.2 of ruby2ruby instead of 1.2.4.

Here's heckle complaining about a function I thought I had fully specified:

The following mutations didn't cause test failures:

--- original
+++ mutation
 def model_matches?(substring)
-  s = substring.gsub(/\./, ".")
+  s = substring.gsub(/\033!\032\002l\}\?V\010d\}\r\-\fyg,a\*jFT\003_"ga\016\020ufN\0066/, ".")
   s = substring.gsub(/\*/, ".*")
   s = "^#{s}$"
   Regexp.new(s, "i").=~(@model)
 end

--- original
+++ mutation
 def model_matches?(substring)
-  s = substring.gsub(/\./, ".")
+  s = substring.gsub(/\./, "\023GA3w+h-#z$?I;a\"k0n^r$\005io#l\023H1M{\034m")
   s = substring.gsub(/\*/, ".*")
   s = "^#{s}$"
   Regexp.new(s, "i").=~(@model)
 end

--- original
+++ mutation
 def model_matches?(substring)
-  s = substring.gsub(/\./, ".")
+  s = nil
   s = substring.gsub(/\*/, ".*")
   s = "^#{s}$"
   Regexp.new(s, "i").=~(@model)
 end

--- original
+++ mutation
 def model_matches?(substring)
   s = substring.gsub(/\./, ".")
   s = substring.gsub(/\*/, ".*")
   s = "^#{s}$"
-  Regexp.new(s, "i").=~(@model)
+  Regexp.new(s, "v\022").=~(@model)
 end

How cool is that?

The only things I've found that are truly difficult to get full test coverage for are tests involving concurrency, esp. race conditions. Trying to prove that a mutex or a critical section must be in place can be difficult. Sometimes you can do it. Sometimes, you just have to shrug your shoulders, put in the line of code you don't know how to test, and move on.