Rails Testing: Fixtures, Factories, and Magic numbers

Question

I've got an application that needs quite a bit of data (1000s of records) to do appropriate testing. The only way I've found to get a decent set of testable, sensible data is to use a subset of my production DB. I've converted this to YAML fixtures in the normal `test/fixtures' location.

This works, but now I have a bunch of seemingly brittle tests and assertions that depend on their being a particular number of records that meet condition X...

example

def test_children_association
  p = Parent.find(1)
  assert_equal 18, p.children.count, "Parent.children isn't providing the right records"
end

This doesn't seem like a good idea to me, but I'm not sure if there is a better / accepted way to test an application that needs a large hierarchy of data.

Answer 1

The first thing I'd say is: what are you testing in that example? If it's an ordinary AR has_many association, then I wouldn't bother writing a test for it. All you're doing is testing that AR works.

A better example might be if you had a very complicated query or if there was other processing involved in getting the list of children records. When you get them back, rather than testing for a count you could iterate through the returned list and verify that the children match the criteria you're using.

Answer 2

what I've found most useful in this situation is not using fixtures at all but rather construct the database objects on the fly like

def test_foo
   project = Project.create valid_project.merge(....)
   *do assertions here*
end

and in my test_helpers I'd have a bunch of methods:

def valid_project
   { :user_id => 23, :title => "My Project" }
end

def invalid_project
   valid_project.merge(:title => nil)
end

I found that the pain of having to build massive collections of test objects has led me naturally to design simpler and more versatile class structures.

Answer 3

Magic numbers in tests aren't an anti-pattern. Your tests need to be so dead-simple that you don't need to test them. This means you'll have some magic numbers. This means that your tests will break when you change small bits of functionality. This is good.

Fixtures have some problems, but there are a few simple things you can do to make them easier to work with:

1. Only have baseline data in your fixtures, the sort of data that most of your tests need but don't care about. This will involve a time investment up front, but it's better to take the pain early than write poor unit tests for the life of the project.

2. Add the data to be tested in the context of the test. This improves readability of your tests and saves you from writing "make sure nobody messed up the fixtures" sanity checks at the beginning of your unit tests.

Answer 4

Cameron's right: What are you testing?

What sort of system needs 1000s of records present to test? Remember, your tests should be as tiny as possible and should be testing application behavior. There's no way it needs thousands of records for the vast majority of those tests.

For little bits of behavior tests where you need object relationships, consider mock objects. You'll only be speccing out the exact minimum amount of behavior necessary to get your test to pass, and they won't hit the DB at all, which will amount to a huge performance gain in your test suite. The faster it is to run, the more often people will run it.

Answer 5

I may have a unique situation here, but I really did need quite a few records for testing this app (I got it down to 150 or so). I'm analyzing historical data and have numerous levels of has_many. Some of my methods do custom SQL queries across several tables which I might end up modifying to use ActiveRecord.find but I needed to get the test running first.

Anyway, I ended up using some ruby code to create the fixtures. The code is included in my test_helper; it checks the test DB to see if the data is stale (based on a time condition) and wipes and recreates the records procedurally. In this case, creating it procedurally allows me to know what the data I'm testing for SHOULD be, which is safer than using a subset of production data and hoping the numbers I calculate the first time are what I should test for in the future.

I also moved to using Shoulda which along with many other useful things makes ActiveRecord Association testing as easy as:

should_have_many :children
should_belong_to :parent