Is it bad form to count on the order of your NUnit unit tests.

Question

I have been creating Unit tests like crazy and find that I'm often having to set up something in one test that I just tore down in a previous test. Is it ever reasonable to create something (e.g. a database record) in one test (e.g. an Insertion test) and then use it for a later test (e.g. a Deletion test)? Or should each and every test always stand completely on its own?

Can you even determine the order of tests in NUnit or are they always done alphabetically?

Note: I am specifically asking about the order of tests within one test file. Not across test files or in any way more globally.

Update: Thanks to everyone that answered - there were a lot of good answers and the sense of the group is pretty unanimous. I've chosen John Nolan's answer as he provided the most complete explanation and lots of links. As you may have guessed, I've been sorely tempted to break this rule despite thinking that it might be a bit "smelly" as John put it. Thanks also to Fortyrunner for adding the unit-testing tag.

Answer 1

I really wouldn't rely on ordering of tests. Instead, I'd pull the common setup code into a separate method and call that from both the simple test and the more complicated one. Alternatively, just call the insertion test itself at the start of the deletion test.

Answer 2

I would view each test as completely independent from any other test. Even if you could mandate the order of the tests, it would be a maintenance nightmare when the tests must change.

Answer 3

Look into test fixture setups that allow you to specify functions that will be executed before any of the tests in the fixture. This allows you to do common setup once and it will always run, whether you run one test, or all tests in the suite.

Answer 4

Unfortunately run order of the unit test is not predictable or at least could be changed in future. E.g. unit testing framework will be changed so each test will be executed in separate thread. So from my point of view using test order is not reasonable. On the other hand you can create a set of small independent tests to test small parts of your code and then create one or several large tests that will run your small tests in specific order.

Answer 5

Unit tests are intended to stand alone, not be run as a sequential script. If you actually need them run sequentially, collect them into a single test function.

If your unit tests suffer from expensive set-up, you may be doing integration testing when you think you're doing unit testing. If you're hitting a SQL database inside most of your unit tests, you're actually integration testing with your data access layer.

Answer 6

I would strongly advise to make all your unit tests independent.

Your business logic / database structure etc. may change over time, so that you'll eventually have to replace or rewrite (or even discard) existing unit tests - and if you have several other tests depending on the one that you're replacing, this might cause unnecessary troubles because you'd have to go through all of the other tests as well and check if these are still working as expected.

In addition, one failing unit test should not be able to drag many others (that might perfectly work on their own) down.

Answer 7

Relying on the order of your tests indicates that you are persisting state across tests. This is smelly

A cleaner way of testing is where you only depend on the single piece of functionality you want to check the behaviour of. Commonly you mock the other objects you need to get your method under test to function.

A good way to think about approaching unit tests is the Arrange, Act, Assert pattern.

Below is a snippet from Karl Seguin's excellent free eBook. I've annoted Arrange, Act and Assert.

[TestFixture] public class CarTest 
{ 
    [Test] public void SaveCarCallsUpdateWhenAlreadyExistingCar()   
    {
         //Arrange
         MockRepository mocks = new MockRepository();
         IDataAccess dataAccess = mocks.CreateMock<IDataAccess>();   
         ObjectFactory.InjectStub(typeof(IDataAccess), dataAccess); 
         //Act
         Car car = new Car(); 
         Expect.Call(dataAccess.Save(car)).Return(389); 
         mocks.ReplayAll(); 
         car.Save(); 
         mocks.VerifyAll(); 
         // Assert
         Assert.AreEqual(389, car.Id); 
         ObjectFactory.ResetDefaults();
    } 
}

Answer 8

If you have stateful tests (a common problem with database work - which is what I do when I'm not on SO), then it seems to me that avoiding order within a test file is not absolutely necessary. However, you have to recognize that if you have 2 tests, with test 2 depending on test 1 passing, then you will get a 'catastrophic' double failure if test 1 fails, because test 2 doesn't have the expected setup (and, what's more, you want to worry if test 2 does pass after test 1 failed if you think that test 2 depends on test 1 passing).

That's why you want the tests independent whenever possible - both intra-file and inter-file.

It would be very unwise to depend on the order between (sets of) tests in different files.