Unit testing huge applications - Proven methodologies?

Question

Hello members

I've been working in windows forms applications and ASP.Net applications for the past 10 months. I've always wondered how to perform proper unit testing on the complete application in a robust manner covering all the scenarios. I've the following questions regarding them -

• What are the standard mechanisms in performing unit testing and writing test cases?
• Does the methodologies change based on the application nature such as Windows Forms, Web applications etc?
• What is the best approach to make sure we cover all the scenarios? Any popular books on this?
• Popular tools for performing unit testing?

Answer 1

You need to write unit tests that test objects and functionality within your program. This means tests to test each function within a class, then the class functionality as a whole, then functionality inside libraries. Essentially you are writing tests for each level in the application right from top level libraries to functions. This ensures that if you change ANY code it will still work as expected.

NUnit is a popular unit testing framework but VS provides you with built in unit tests as standard (might be VS2008 only but pretty sure VS2005 does too)

Covering all scenarios is simply a case of writing tests to cover all known eventualities, you cannot cover EVERY scenario but you can cover the main ones, which normally are fine as you know what output is expected and can test against that.

Answer 2

If you have already been developing an application for 10 months it is highly unlikely that you will be able to unit test it at all, because unit testing per definition involves testing only one unit (class or even method) at a time.

At this point, you would be better off writing automated acceptance tests using such tools as FitNesse or StoryTeller.

Answer 3

I recommend a very good book on this subject: Working Effectively with Legacy Code by Michael Feathers. I found it immensely useful for similar legacy projects.

The main problem with legacy code is that there is no "standard" way to unit test it :-( The "standard" Test Driven Development is invented for new projects, where you start writing code - and unit tests - from scratch, so you can grow your unit test suite together with your code from day 1, and keep all (or most) of your code covered all the time.

However, reality is that most of the real life projects involve legacy code, without a single unit test (in fact, Feathers' definition of legacy code is "code without unit tests"). This book is full of useful advice on what to do when you need to touch code you barely understand, or modify a monster method of 1000 lines and make sure that at least your modification gets unit tested properly. In such cases, typically it is very difficult to write unit tests, because the code was not designed to be testable. So often you have to refactor it to make it testable, but of course without unit tests in place, this is risky... still, there are ways out of such traps, and this book shows them.

In general, you shouldn't start with the aim of covering the whole codebase (unless you have a project manager willing to accept that you are not going to produce any new feature in the next couple of months - or years ;-). You have a limited amount of time to achieve the most possible benefit from your unit tests. Thus you have to focus on the most critical parts of the code first. These are typically the ones most often modified and/or the ones where the most bugs are found (there is a correlation of course). You may also know in advance that an upcoming feature requires extending a specific part of the code, so you may prepare the way by creating unit tests for it. This way, over time you start to grow little "islands of safety" within the code, which are ever better covered with unit tests. Maintenance and refactoring is easier in these spots, and as you add more unit tests, the islands slowly grow...

Note that in the beginning these "safe islands" don't tend to show a very "systematic" pattern of occurrence. The most critical, most often modified parts in the code usually are distributed fairly randomly. Only at a much later stage, when the unit tested islands start to grow and merge, is it worth covering a specific module more systematically. E.g. if you see that in this particular module the code coverage of unit tests grew over 60%, you may decide to go through it and add tests for the remaining code parts too.

Answer 4

If you want to know how to test whole application, it's not a unit test, since the unit when talking about .Net applications is a class or method. I guess your confusion arouse from the fact that the automated thing you create is a test unit (you do unit tests with tests units, but you can also do integration tests with test units...). You are probably talking about automated integration tests or acceptance tests (which should never be automatic).

Testing models such as V-Model define at least three test phases:

Unit Testing

Tests a single functionality or feature of the system. It's based on the technical specification of the unit (class or method) that you are building. It can be automated through the use of tests units. Also, I think one should use a CI Server (continuous integration) here, since if your units don't correctly integrate, there problem has most likely appeared in this phase.

Integration Testing

Once you certified that each of your "units" (classes or methods) works individually, you try now to check if the whole system is working as one. So you do an integration test, which checks to see if those units work correctly together in order to achieve the purpose of the system. You should try to automate Integration Tests as well. You can use tools like StoryTeller for that.

User Acceptance Testing

User Acceptance test must be conducted by the user, so no automation here. Of course, you can create and load the data that the user will validate, so portions of the tests are automated, but not the result. No robot should ever give you the final word that the system is working, only the user.

Now, to answer your questions:

What are the standard mechanisms in performing unit testing and writing test cases?

You should use .Net Test Units for that purpose. In order to test user interaction (you can test a user interaction with a screen), you can use a external application (check the links below). Some robot might automate such tests to you.

Does the methodologies change based on the application nature such as Windows Forms, Web applications etc?

Indeed. For example, a batch application can be unit tested through the sole use of test units. Tests cases for batch systems can also be more lax, but will need more data input to check all the constraints.

What is the best approach to make sure we cover all the scenarios? Any popular books on this?

I'm not sure that there's a real way to make sure that you covered every scenario, but what you should do is define what you are testing against. Like I said before, you should unit test against the technical specification, so if your spec is well written, you should be able to clearly identify the test cases. If you feel the need to test against something that wasn't specified, then you should be improving your design techniques.

Popular tools for performing unit testing?

List of GUI testing tools

List of unit testing frameworks

Hope this helps.

Answer 5

What are the standard mechanisms in performing unit testing and writing test cases? There are many .NET testing frameworks. I would recommend you give NUnit or MbUnit.

http://www.nunit.org/

http://www.mbunit.com/

MbUnit will come with a test runner, so that might make things easier. I prefer an integrated approach where I can run all my tests right from visual studio, but that requires some significant setup. When using a test runner (such as Gallio which comes with MbUnit) you will be running your unit tests external of your visual studio project. Unit Tests themselves should be located within their own project, generally of the form Fixture. The tests within can be named with differing styles, but the key is to be as descriptive as possible. For example:

void when_my_class_is_sent_a_user_it_should_save_it()

Also popular

[MethodName_StateUnderTest_ExpectedBehavior]

Does the methodologies change based on the application nature such as Windows Forms, Web applications etc? The methodology does change based on the applicaiton nature. Fortunately, web forms and win forms are very similar. Unfortunately, they both were designed without testing in mind and tend towards code that is difficult to test properly.

What is the best approach to make sure we cover all the scenarios? Test Driven Design (You write the tests, then write the code to make them pass). Code coverage can also be a helpful metric.

Any popular books on this? The Art of Unit Testing: With Examples in .Net By Roy Osherove Pragmatic Unit Testing in C# with NUnit

Popular tools for performing unit testing? As mentioned above, NUnit & MbUnit. There is also MSTest (comes bundled with VS2008), xUnit and some others. Stongly suggest you go with NUnit or MbUnit though.