The first real software company that I worked at was all about the unit testing (NUnit). I don't know that we were real sticklers for it back then -- I have no idea what our code coverage was like and I was writing most of the unit tests. Since then I've run into some companies that do lots of testing, but it's chair testing: relies on a person being there, has low repeatibility and low chance of catching bugs. The other attitude is: it was something they wanted to get going with "in the future"; basically when money falls from the sky.

I miss unit testing -- it just makes life easier. But I'm finding that when I look for a new job, unit testing is either something that companies would like to "get going with" in the future or something they don't do at all (uhh, it's been around for a while now!). I'd say that 60-75% of the job reqs I've looked at over the past 2 years have not listed unit testing at all. I can only think of one or two that had unit testing experience as a requirement (for a mid-level developer position).

So the question is, what's missing? I think it makes people more productive, but that's only after spending a solid amount of time actually doing it. Aren't there any good studies about the cost savings of unit testing? Is it the type of company I'm looking at?

Edit: even though the title is a bit devils-advocate, I consider myself a unit testing proponent.

+9  A: 

There are plenty of companies out there that really do nothing along the lines of best practices. No code reviews, no unit testing, no testing plans, no nothing, just by the seat of their pants.

Take this as an opportunity to get them to use a Continuous Integration platform and develop unit tests! Easy way to impress the powers that be and increase the quality and stability of your code at the same time

Edit: As for reason, I think they just plain aren't aware of the current tools out there that make CI and unit testing extraordinarily easy.

I'm typically in a position where I have zero influence over policy decisions like these; I'm junior senator being overruled by committee chairs.
It takes a few minutes to start Hudson and write unit tests. If unit tests are already on your "TODO" list, then you're just doing your job. Committees are often impressed by fancy charts and images, so show them some pretty trend graphs from Hudson;)
Yeah! Let's hear it for the truth. Despite all the time we developers spend banging on about best practice, it's not always used in the real world. Pity really.
Chris Needham
+4  A: 

Its probably a combination of a couple of things you mentioned already. Its difficult to measure cost savings of TDD. If you want to outsource your IT, you can show how much you pay per year for the guys you have on staff vs. the cost of contracting it out; its very concrete. How do you say, "Oh, this test caught a bug which would have taken me 4 hours to debug and fix..."?

How on earth can you guess how long a bug would take to debug and fix. Generally debugging is more random than that in my experience - I happen to have an idea where the problem is or I don't.
Dominic Rodger
Yeah, that's why I said its so hard to quantify the benefits of testing.
+52  A: 

1) It's hard
2) It takes time
3) It's very hard to determine the value of test code

Point 3 is a sticky one. Good unit tests reduce bugs. But so does good production code. How do you determine how many bugs don't exist because of your unit tests? You can't measure what does not exist. You can point to studies, but they don't fit nicely on your business manager's spreadsheet.

"Good unit tests reduce bugs. But so does good production code." -- Good unit tests make it possible to improve the production code. Even if the code is at first bad, but you have good test coverage, you can confidently refactor the system until the code is good.
Esko Luontola
Esko, in addition to having good coverage, you also have to have good tests that actually test something. You could have 100% code coverage and actually be testing very little
Jacob Adams
Excellent answer! You've said the same thing as my answer in far fewer words.
+1 nice answer; well put :-)
Yes, unit tests take time. But so does "random bug fixing". Proper test driven development has "all" features "documented" in tests. If the tests are green, the product works as intended (except for usability issues etc). My experience is that the total development time is almost not affected at all. You spend more time on things, and get them right the first time around rather than after time spent on bug fixing.
Arve Systad
+67  A: 

In my experience, there are a couple of factors involved in this:

  1. Management doesn't really understand what unit testing really is, or why it has real intrinsic value to them.
  2. Management tends to be more concerned with rapid product delivery, and (incorrectly) sees unit testing as counterproductive to that goal.
  3. There's a misperception that testing belongs solely in the pervue of QA. Developers are coders, and can't write tests.
  4. There's a common misperception that management will have to spend money to do unit testing correctly, despite the fact that the tools are freely available. (There is, of course, the developer ramp up time to consider, but it's not really prohibitive.)
  5. Will's answer will round this answer out: It's very hard to determine the value of test code (edit jcollum)

Naturally, there are other factors, but those are just what I've run into so far.

Mike Hofer
Yea, pretty much described my management and we have no testing.
And support for it in some popular langauges (C/C++) is poor.
Martin Beckett
@mgb - CxxUnit works pretty well for me...
Dominic Rodger
CxxTest is very good as well. Due to the poor reflection mechanisms in C++, it seems there are more varied "solutions" presented; CxxTest requires a preprocess step in the build system, whereas tools like TUT are entirely compile-time and language-supported, but awkward to use.
Coders rarely manage and managers rarely code.
Tim Matthews
I have found that users on stackoverflow.com tends to blame management for a lot of their problems like these. When I have asked my real-life friends about their 'management issues' that arise, usually I have found that they have never even voiced their concerns to management, must less embarked on a campaign to change people's viewpoints. Instead of saying 'management doesn't...' I try and think of ways I can help management see my point of view, and convince them my position is correct. I think this is a problem because developers aren't good at selling unit test.
Brian Stinar
+13  A: 

There have been studies done on the ROI of unit tests - see this question.

Dominic Rodger
good link, thanks
+2  A: 

In my experience it really depends on the software you are writing. I have found it is extremely hard to write unit tests for a UI. I only use unit tests for portions of the system that have a definite in/out.

Shaun Bowe
I agree. If you use model/view/controller it makes a lot of sense to unit test the model and the controller. UI is almost always best tested by a human.
Zan Lynx
+4  A: 

The reason some places don't use it is simply because it takes a lot of work both to get started and to continue. The fact that writing unit tests takes about as much time as writing the actual functionality seems to some managers like you're cutting your developer's productivity in half.

On top of that, you build team (or someone) needs to put the infrastructure in place and maintain it.

And as Alan says, a lot of places simply don't use best practices - they just want to see something tangible.

Michael Burr
+10  A: 

I've found a lot of developers who aren't interested in unit testing. It always seems like a lot of work with little payoff when you start. No one wants to sign up for extra work and so they resist. Once people start, they usually enthusiastically stick with it, but getting them started can be hard.

Steve Rowe
+4  A: 

Unit testing should be just a natural part of the code development workflow, just as the compiler is.

However, this requires educating the management on the benefits of the unit testing. Junior developers have relatively low chances to have such influence, though. Thus, whether a company is a proponent of the unit testing depends on whether they have a senior developer or architect that is an advocate of unit testing.

I believe this is the answer to your question "what's missing and why aren't more companies doing unit testing". :-)

Franci Penov
+1 for "should be a natural part of the code development workflow". Every *professional* developer should be doing there own *unit* testing regardless of official process. The only legitimate argument on this issue is the definition of a *unit*.
@Dunk If you don't define "unit" then you're only saying that "professional developers should be doing their own testing".
@ChrisW - Yes, professional developers should be doing their own testing. There is no reason for a developer to ever turn in code that they have not tested enough to feel confident that it works correctly. Unfortunately, this appears to be too much to ask of many developers.
When I say the definition of a unit is a legitimate argument, I am talking about the granularity of a unit. Is it a class? Is it a collection of classes? Is it a component? I think class level unit testing (with some exceptions) costs more than it benefits and leads to many meaningless tests...
which others have pointed out in this thread. Whereas, if one defines a collection of classes that work together as a unit then you can still do automated testing and your tests are generally more meaningful because they can focus more on higher level required functionality.
+1  A: 

I miss unit testing -- it just makes life easier.

That is not, really, a sufficient reason for a company to adopt unit testing.

A sufficient reason might be "cheaper" (and/or, "better"): which is not as easy to prove about unit testing.

The only good reason might be "writing unit tests is the best use of developers' time", which is really hard to prove IMO: and it may be true in some places, for some software, with some developers, and not true elsewhere.

There are plenty of developers who don't think the world of unit tests: including some who think that other forms of testing (e.g. automated integration/functional tests) may be cheaper and more valuable, for example Am I the only dev who doesn't like unit tests?

+2  A: 

People are lazy and only adopt change when forced to.


Simple it cost money to write and update unit tests. Most of companies previous software doesn't have unit tests and will cost too much to write. So they do not do it and it adds time to the development process so they also do not add it to new features.

David Basarab
You should read the links about ROI.
+2  A: 

Most companies are useless. Not the one that you (or I) work for, obviously.

Roger Lipscombe
+18  A: 

Most tests don't test anything.
You write a fileopen() function and a unittest that fails if the file doesn't exist and succeeds if the file does exist. Great! Now did you check if it works with the filename in BIG5 chinese? on an NFS share? on vista with the file on a USB key and UAC turned ON?

The problem is that the unit tests are written by the same programmer who wrote the function, to the same set of assumptions and with the same level of skill. To really work the tests must be written by someone else, only to the published specs without them seeing the code. - In most companies just getting written specs would be a breakthrough!

Unit tests check for errors in the code of individual functions. They can work for data access layers, maths libraries etc where the inputs/outputs are well known and the internal structure is complex but for a lot of cases they are just a waste of time.
They fail when the errors are due to interactions between different parts of code or with the OS and the user. Problems like high/low DPI settings messing up a dialog box or a foreign language setting swapping a '.' and ',' aren't usually found.

Martin Beckett
I think this answer misses the mark a bit. Unit testing and functional testing are not, and should not be, the same thing.
An element you are missing is that unit tests are not just a one-and-done thing. If I find out later that I need to fix a bug with fileopen() on an NFS share, then I can add a test to my test suite for this. Then, when I do more development in the future I have regression testing in place.
Paul Osborne
Many errors come form interactions outside the code that the programmers hasn't thought of and can't be found by simply checking the code. A common gui problem is machines with very high/low DPI settings - you can unit test the dialog function all you like but won't spot this.
Martin Beckett
that's not the function of a unit test though, and interaction between different parts of code is very unit-testable and indeed, if those parts are being written by separate teams, unit testing your code against a shared interface and mocking the other team's component are good practices.
TDD handles the rigged test issue of "the programmer that wrote the code also then writes the tests" by making you write the tests before you write the code.
I think that makes it worse - you have a set of assumptions, you code those into the test then write a function that matches that test/assumptions.
Martin Beckett
@Martin Beckett (I know I'm a year late to the party): yes, you code in a circle from your assumptions to your assumptions. But that does seem at least minimally helpful because as your code changes, you must always line up assumptions (axioms) with assumptions (unit tests). I think that being forced to draw that line could help.
+8  A: 

Aside from the issue of adoption of unit testing, unit testing isn't always worthwhile, though in general I think it is, when applied properly. There is nothing special about unit tests that saves them from being vulnerable to poor construction.

Unit tests have costs (creation, maintenance, and running) and are only worthwhile if they provide greater benefits than those costs. Test creation is a skill like any other, it requires specific experience and knowledge for success. Without sufficient experience it's very easy for even otherwise experienced developers to create low quality, low value, and/or high cost unit tests which are not worthwhile. Especially so given how difficult it can be to judge the value of a unit test.

Additionally, unit testing is just one way to improve code quality, but it's not the only way. In some circumstances and with some teams it may not be the most effective way to increase the quality of software.

Keep in mind that putting a lot of effort into unit testing is no guarantee of quality software. And, too, it is possible to produce software of the highest quality without any unit testing whatsoever.

What you are saying is true in statically typed languages. In dynamically typed languages they are absolutely critical. I mean your code is almost guaranteed to be crap without the tests. I find this to be a large part of why some people seem to value unit tests so highly.
Bill K
+1  A: 

Like most good ideas, adoption has more to do with organizational path dependence than with the quality of idea.

In most companies that have shipped products, a substantial QA division has been created with a senior level QA head. Testing is the fiefdom of the QA team.

The QA team is unlikely to write unit test code because the company typically doesn't staff the QA team with its heavy duty coders.

The programming team is reluctant to write test code because it creates a conflict with the QA team.

I've been seeing more interest and adoption of Unit Testing in groups where QA hasn't been spun off into a separate job funtion

+47  A: 
Good idea. But don't call out the time to create unit tests as separate from the time to create the "product" code!
Jeff Kotula
Reading this answer made me realise that, While I'm familiar with the concept and the basics of unit testing, I really don't know how to do it effectively at all. Can you reccomend a good book on the subject?
Management doesn't specifically say "don't unit test," but they do hound you and ask for progress reports on cranking out features. Sometimes they are not willing to listen to all the progress you made on unit tests. Just an observation.
Andy White
On one kernel driver that I worked on, I refactored a bunch of code into a library that I linked into a user-mode test harness. This particular code was environment-agnostic, so it was easy enough.I haven't tried it, but IRPs, like filesystems and databases, should be mockable.
George V. Reilly
With Bochs or QEMU it is possible to write a simulation of your device for your kernel driver to speak with.
Zan Lynx
@Breton - Check out The Art of Unit Testing by Roy Osherove
Ola Eldøy
@floding, fascinating answer. I think I'll have to go read a book on unit testing.
+2  A: 

I think part of the problem is that developers are expecting business people to have the same set of values and to really care about the answer to "should we unit test or not?". We don't get approval beforehand from the business to use a high-level language rather than assembly language -- it's just usually the sensible way to get the work done.

The point is, we are the only ones qualified to make the call (which isn't to say that all of us have the same knowledge on the topic). Furthermore, even if your team doesn't, as a matter of policy, do unit testing (or name-your-method-of-the-day) it generally doesn't mean that you can't do it.

The truth is, we can't really prove ROI on most of the stuff we do at too fine of a granularity. Why unit testing is held up to this unreasonable/non-typical standard of proof is beyond me...

Jeff Kotula
You do need management involved, however, because you have to get your co-developers on board and the best way for that to happen is for it to be a requirement from the top down.
+1  A: 

Two things are obstacles to unit testing

  1. Everything new is hard to do
  2. Everything that has no measurable profit, is bad.
  3. Humans are lazy. Developers, indeed.

In my company (> 5.000 emp) unit tests "are there", but no chance to do TDD or get a big code coverage. It's to hard to accomplish it.

+3  A: 

From what I've seen, a lot of companies have huge, highly-coupled code bases that aren't practically unit testable. They also don't have decent testable requirements, so the unit tests would test against "as built" de facto requirements.

Rob K
+6  A: 

Well, my company hasn't gone with TDD or Unit Testing. To be honest, we're not sure how to do it. We can obviously do it for stupid functions like CapitalizeString(), etc, but we don't know how to do it for highly complex systems with complicated objects. Moreover, most of the people interviewed have either zero experience, or limited experience. It appears that Unit Testing is big from the SO crowd, but not particularly big in the available workpool.

TDD is a separate topic. We are morally opposed to TDD. We aren't cowboy coders, but we do believe that it stunts creativity and flexibility in a project. Moreover, having the coder who wrote the unit testing function makes no sense. When I do something, I code to all of the edge cases I can think of. What I need is another brain to look for things I might have missed. We don't have that. The teams are small and self-contained.

In short, we don't believe in TDD, but we would like to Unit Test. We just don't have the experience to do so, and we can't find it easily.

Back when the place I work had enough coders for it, we did pair programming. I found it very effective for one to write tests as the other coded. It led to very intelligent questions about the code, too.
Zan Lynx
The point to TDD, which you seem to be missing, is that you write all of the edge cases into your unit test. For each case, you write an assertion in your unit test. Then, if your actual code fails an assertion, you know that your code has a bug in it, and you've implemented your logic incorrectly. Since units are small, and your assertion tests specific code in the unit, it should be easy to pinpoint where your logic error is. The important thing is that you write your assertions first. Then make your code pass. Can you point out where this process stunts anything but bug-growth?
Christopher Parker

I know one person who will look at a form with a single button and say "That's overkill, this should have a command line interface, so it can be automated in a script" and yet he never really got behind unit tests. I just don't get it. Fear of change could be part of it, but I've offered to walk him through the process and he's declined.

Obviously my current company doesn't enforce them, I've just taken it upon myself to write them when possible. I have found that setting up tests up front leads to witting better/cleaner code, because in thinking how to test something you generally have to consider it's later usage.

+1  A: 

Of course, in the ideal world, you can not argue against having a unit test.

However, whether you write a unit test depends on a number of things:

  • How the software will be used. If you were writing software for just yourself would you write unit tests? Probably not. If you were writing pre-packaged software to be sold commercially, probably yes.

  • How many people maintain the code....if it's just you, then you may know it well enough to be confident enough after making a change that a quick run through of the code is sufficient to ensure nothing has broken. If other people who did not originally write the code must now maintain it then a unit test will help give them confidence that when they update the code to fix a big (that was obviously not captured in the unit test!) they have not broken anything.

  • code complexity: only test code which needs a test. A one line variable assignment method does not need testing. A 50 line method with multiple paths of execution probably does.

  • Practical commercial commercial considerations: The fact is writing unit tests does take longer than not doing so. If you are writing prototype software, that has an uncertain commercial future, then there is a pay off to be has between having code quickly, now, that works sufficiently well versus having unit tested code in 2 weeks that works better. Sometimes it pays to find out quickly (consumer appetite) if software will have a short shelf like and move on to the next project.

and as others have pointed out, a test is only as good as the person that wrote it.

+1  A: 

I've worked for 3 companies so far, and have just recently started writing "offical" NUnit tests, and I have to say I'm a big fan. There are 2 points I've noticed however in the last few weeks:

  1. It depends on the application - there are certain apps/pieces of an app that can be very easy to unit test. A function that does something basically the same each time, and produces an easily recognizable result. And there are sections that don't do this: a block of code that prints crystal reports or modifies an image - here the eyeball is actually needed and probably better/faster. These areas can discourage people from trying to write tests. They then just start thinking that if certain areas can't be unit tested, why bother?

  2. The granularity of the test- many DEVs/QAs already have some sort of automation in place to 'test' a piece of functionality, and get confused as to how this is different from a unit test. Depending on their test, it might not be. The important thing here that takes some time to appreciate is that a good unit test is totally granular. It tests the minimum piece of logic that makes sense, which makes it repeatable, automatic, and hopefully always valid. It took me a little bit to really appreciate how useful that is, especially when you have a huge codebase and like to run regression tests after making a change to see if you just foobared the whole app.

+1  A: 

I think it's worthwhile to write a good unit test coverage for the code that you gonna be working with way into the future. For example, when you are working for a product company, by writing quality unit test cases you are simply making a time investment, knowing that all the hard work writing test cases will pay off and make your life easier down the line.

The sad truth is that a lot of software programmers work on in-house application development or contracting and just trying to get some software out the door. In those cases, the time investment does not make that much sense, as chances are you never see this pile of code ever again.

We can all argue that adhering to solid software development practices is responsibility of every developer. However, ROI means a lot to individual developers as it does to management.

Rocket Surgeon
+1  A: 

This is going to sound a little like an AA meeting.

I am a software engineer.

I was talking about software engineering to my brother at XMAS (he is a business analyst).

  • He has been working as a BA for 10 years.
  • The conversation got around to testing.
  • He had NEVER seen (or heard of) automatic unit testing.
  • Or even heard of developers doing it.
  • In that time he was worked for some of the big "alphabet" companies, on large government contracts.

So as far as I could tell, of the 100's of developers involved, none of them had heard of unit testing.

My predecessors in my current job used automatic validation tests, and had very few unit tests. Needless to say the code needs heavy refactoring.

Tim Williscroft
+1  A: 

If you want to sell everyone on testing do the following:

  1. Write a bunch of tests.
  2. Notify the other developers who change code and fail your test(s).
  3. They'll fix their code.
  4. Now you can release without those particular bugs.

Even a manager could understand this.

Jeff O
Unit testing won't cause your release to be bug-free. It may reduce the bug count, but it's awfully hard to measure the number of bugs that /didn't/ hit production because of testing.
I don't know that management would be happy with someone writing a bunch of tests when they could be coding new functions. If they aren't on board with TDD you're likely to get in trouble for writing tests to cover code you didn't write.
@jcollumAs usual, it depends on the cost/profit ratio.
+3  A: 

I don't think laziness is the root cause of bad unit testing. For my company, time constraints and "just get it done" attitude are the biggest deterrents for doing unit testing. Also, the places where our systems fail tend to be more at the integration level (services, database accesses, complex queries that require specific data for testing), not the "unit level." These things are just harder to test, and if you barely have enough time to get the feature done, you're probably not going to have time to get any useful tests done at the same time.

Andy White
This is common. I think the argument is that if you think you'll ever change it in the future then you should have a test that makes sure it's working after it changes.
+2  A: 

My 2 cents:

  • It requires some education and discipline, but new graduates already come with the proper knowledge.
  • Tests overhead can be reduced with better tools and this is happening also (refactoring etc.)

So, it's just a matter of time.

There's the Martin-Coplien debate in which Bob Martin asserts that:

"nowadays it is irresponsible for a developer to ship a line of code he has not executed in a unit test."


good link, thanks.
I do not believe in the existence of a complex, real world system in which every single line of code was covered by unit tests.
+3  A: 

Unit testing is one of those black box terms that most people have heard, but don't know what exactly constitutes a unit test, where to start, how to write them, how to actually run tests, what exactly they should be testing, etc. etc. etc.

In a lot of cases, its easier for the unsure developer to simply dismiss them as unnecessary or some icing that only "enterprise level developers" need.


The unit testing is great (easy to implement, easy to determinate that you have wrote enough test examples) when you have some code that operates as a server.

Example 1: You should mark in database posts that have 3 or more site moderator reviews What are tests variants

  • check post without comments -> expect no marks
  • check post with one comment -> expect no marks
  • check post with 3 comments by common users -> expect no marks
  • check post with 3 comments by common users and at least 1 comment by moderator -> expect no marks
  • check post with 3 comments by common users and at least 3 comments by moderator -> expect mark for this post


Example 2: You have a web site page that output the list of posts

How we should check that this list is misaligned in some browser? How we should check that this list is colored properly and we didn't crush styles?

there is no easy solution to automate this

so the task during unit test creation is to understand - can you put code in server operation mode - if you can create test, if not then break to pieces and create test for the pieces or test manually.

And one more thing - the economy of unit testing you will get if you have more than 10-12 testing stages without significant code modification.

Regards, Pavel

+1  A: 

I think that the programmer has to just start doing it. A few simple tests to start with are easy to justify as part of development.

Something like a unit test is almost always necessary to get fast debugging turn around. Just explain how much faster it is to launch the test than it is to arrange the correct input, set a debugger breakpoint, launch the application, etc.

Document the test in your code. Just put a comment explaining where the test is and how to run it. Future programmers will see it and hopefully the testing will spread!

Zan Lynx
+1  A: 

Companies are not performing unit testing, for the same reason that many websites are written poorly – ignorance, and people sticking to old habits. At my company, since we started unit testing (with Nunit, and Typemock), we reach higher code coverage and release the software in a shorter time to market.

+3  A: 

The main reason is that many developers and development managers don't have a clue that unit tests exist, or how to use them.

The second reason is that unit tests can only be used (in a sensible manner) with code that already fulfills some quality standards. Chances are, that some existing codebase doesn't fall into that category.

The third reason is lazyness and/or cheapness.


I think the main reason is simply that most managers and developers just don't care enough about quality. So they tend to believe that writing and automating unit tests is just too much effort (which it actually is) for little benefit.

Maybe there is not much we can do about the benefits, but we should be able to reduce the costs of creating and running tests by improving the relevant tools.

I find most current open source tools for developer testing to be quite primitive. Unfortunately, there doesn't seem to exist much interest in the industry to improve this situation.


Because unit tests are only usefull if you write testable code. And writing testable code is hard. And people are lazy.