If unit testing is so great, why aren't more companies doing it?

Question

The first real software company that I worked at was all about the unit testing (NUnit). I don't know that we were real sticklers for it back then -- I have no idea what our code coverage was like and I was writing most of the unit tests. Since then I've run into some companies that do lots of testing, but it's chair testing: relies on a person being there, has low repeatibility and low chance of catching bugs. The other attitude is: it was something they wanted to get going with "in the future"; basically when money falls from the sky.

I miss unit testing -- it just makes life easier. But I'm finding that when I look for a new job, unit testing is either something that companies would like to "get going with" in the future or something they don't do at all (uhh, it's been around for a while now!). I'd say that 60-75% of the job reqs I've looked at over the past 2 years have not listed unit testing at all. I can only think of one or two that had unit testing experience as a requirement (for a mid-level developer position).

So the question is, what's missing? I think it makes people more productive, but that's only after spending a solid amount of time actually doing it. Aren't there any good studies about the cost savings of unit testing? Is it the type of company I'm looking at?

Edit: even though the title is a bit devils-advocate, I consider myself a unit testing proponent.

Answer 1

There are plenty of companies out there that really do nothing along the lines of best practices. No code reviews, no unit testing, no testing plans, no nothing, just by the seat of their pants.

Take this as an opportunity to get them to use a Continuous Integration platform and develop unit tests! Easy way to impress the powers that be and increase the quality and stability of your code at the same time

Edit: As for reason, I think they just plain aren't aware of the current tools out there that make CI and unit testing extraordinarily easy.

Answer 2

Its probably a combination of a couple of things you mentioned already. Its difficult to measure cost savings of TDD. If you want to outsource your IT, you can show how much you pay per year for the guys you have on staff vs. the cost of contracting it out; its very concrete. How do you say, "Oh, this test caught a bug which would have taken me 4 hours to debug and fix..."?

Answer 3

1) It's hard
2) It takes time
3) It's very hard to determine the value of test code

Point 3 is a sticky one. Good unit tests reduce bugs. But so does good production code. How do you determine how many bugs don't exist because of your unit tests? You can't measure what does not exist. You can point to studies, but they don't fit nicely on your business manager's spreadsheet.

Answer 4

In my experience, there are a couple of factors involved in this:

1. Management doesn't really understand what unit testing really is, or why it has real intrinsic value to them.
2. Management tends to be more concerned with rapid product delivery, and (incorrectly) sees unit testing as counterproductive to that goal.
3. There's a misperception that testing belongs solely in the pervue of QA. Developers are coders, and can't write tests.
4. There's a common misperception that management will have to spend money to do unit testing correctly, despite the fact that the tools are freely available. (There is, of course, the developer ramp up time to consider, but it's not really prohibitive.)
5. Will's answer will round this answer out: It's very hard to determine the value of test code (edit jcollum)

Naturally, there are other factors, but those are just what I've run into so far.

Answer 5

There have been studies done on the ROI of unit tests - see this question.

Answer 6

In my experience it really depends on the software you are writing. I have found it is extremely hard to write unit tests for a UI. I only use unit tests for portions of the system that have a definite in/out.

Answer 7

The reason some places don't use it is simply because it takes a lot of work both to get started and to continue. The fact that writing unit tests takes about as much time as writing the actual functionality seems to some managers like you're cutting your developer's productivity in half.

On top of that, you build team (or someone) needs to put the infrastructure in place and maintain it.

And as Alan says, a lot of places simply don't use best practices - they just want to see something tangible.

Answer 8

I've found a lot of developers who aren't interested in unit testing. It always seems like a lot of work with little payoff when you start. No one wants to sign up for extra work and so they resist. Once people start, they usually enthusiastically stick with it, but getting them started can be hard.

Answer 9

Unit testing should be just a natural part of the code development workflow, just as the compiler is.

However, this requires educating the management on the benefits of the unit testing. Junior developers have relatively low chances to have such influence, though. Thus, whether a company is a proponent of the unit testing depends on whether they have a senior developer or architect that is an advocate of unit testing.

I believe this is the answer to your question "what's missing and why aren't more companies doing unit testing". :-)

Answer 10

I miss unit testing -- it just makes life easier.

That is not, really, a sufficient reason for a company to adopt unit testing.

A sufficient reason might be "cheaper" (and/or, "better"): which is not as easy to prove about unit testing.

The only good reason might be "writing unit tests is the best use of developers' time", which is really hard to prove IMO: and it may be true in some places, for some software, with some developers, and not true elsewhere.

There are plenty of developers who don't think the world of unit tests: including some who think that other forms of testing (e.g. automated integration/functional tests) may be cheaper and more valuable, for example Am I the only dev who doesn't like unit tests?

Answer 11

People are lazy and only adopt change when forced to.

Answer 12

Simple it cost money to write and update unit tests. Most of companies previous software doesn't have unit tests and will cost too much to write. So they do not do it and it adds time to the development process so they also do not add it to new features.

Answer 13

Most companies are useless. Not the one that you (or I) work for, obviously.

Answer 14

Most tests don't test anything.
You write a fileopen() function and a unittest that fails if the file doesn't exist and succeeds if the file does exist. Great! Now did you check if it works with the filename in BIG5 chinese? on an NFS share? on vista with the file on a USB key and UAC turned ON?

The problem is that the unit tests are written by the same programmer who wrote the function, to the same set of assumptions and with the same level of skill. To really work the tests must be written by someone else, only to the published specs without them seeing the code. - In most companies just getting written specs would be a breakthrough!

Unit tests check for errors in the code of individual functions. They can work for data access layers, maths libraries etc where the inputs/outputs are well known and the internal structure is complex but for a lot of cases they are just a waste of time.
They fail when the errors are due to interactions between different parts of code or with the OS and the user. Problems like high/low DPI settings messing up a dialog box or a foreign language setting swapping a '.' and ',' aren't usually found.

Answer 15

Aside from the issue of adoption of unit testing, unit testing isn't always worthwhile, though in general I think it is, when applied properly. There is nothing special about unit tests that saves them from being vulnerable to poor construction.

Unit tests have costs (creation, maintenance, and running) and are only worthwhile if they provide greater benefits than those costs. Test creation is a skill like any other, it requires specific experience and knowledge for success. Without sufficient experience it's very easy for even otherwise experienced developers to create low quality, low value, and/or high cost unit tests which are not worthwhile. Especially so given how difficult it can be to judge the value of a unit test.

Additionally, unit testing is just one way to improve code quality, but it's not the only way. In some circumstances and with some teams it may not be the most effective way to increase the quality of software.

Keep in mind that putting a lot of effort into unit testing is no guarantee of quality software. And, too, it is possible to produce software of the highest quality without any unit testing whatsoever.

Answer 16

Like most good ideas, adoption has more to do with organizational path dependence than with the quality of idea.

In most companies that have shipped products, a substantial QA division has been created with a senior level QA head. Testing is the fiefdom of the QA team.

The QA team is unlikely to write unit test code because the company typically doesn't staff the QA team with its heavy duty coders.

The programming team is reluctant to write test code because it creates a conflict with the QA team.

I've been seeing more interest and adoption of Unit Testing in groups where QA hasn't been spun off into a separate job funtion

Answer 17

I think part of the problem is that developers are expecting business people to have the same set of values and to really care about the answer to "should we unit test or not?". We don't get approval beforehand from the business to use a high-level language rather than assembly language -- it's just usually the sensible way to get the work done.

The point is, we are the only ones qualified to make the call (which isn't to say that all of us have the same knowledge on the topic). Furthermore, even if your team doesn't, as a matter of policy, do unit testing (or name-your-method-of-the-day) it generally doesn't mean that you can't do it.

The truth is, we can't really prove ROI on most of the stuff we do at too fine of a granularity. Why unit testing is held up to this unreasonable/non-typical standard of proof is beyond me...

Answer 18

Two things are obstacles to unit testing

1. Everything new is hard to do
2. Everything that has no measurable profit, is bad.
3. Humans are lazy. Developers, indeed.

In my company (> 5.000 emp) unit tests "are there", but no chance to do TDD or get a big code coverage. It's to hard to accomplish it.

Answer 19

From what I've seen, a lot of companies have huge, highly-coupled code bases that aren't practically unit testable. They also don't have decent testable requirements, so the unit tests would test against "as built" de facto requirements.

Answer 20

Well, my company hasn't gone with TDD or Unit Testing. To be honest, we're not sure how to do it. We can obviously do it for stupid functions like CapitalizeString(), etc, but we don't know how to do it for highly complex systems with complicated objects. Moreover, most of the people interviewed have either zero experience, or limited experience. It appears that Unit Testing is big from the SO crowd, but not particularly big in the available workpool.

TDD is a separate topic. We are morally opposed to TDD. We aren't cowboy coders, but we do believe that it stunts creativity and flexibility in a project. Moreover, having the coder who wrote the unit testing function makes no sense. When I do something, I code to all of the edge cases I can think of. What I need is another brain to look for things I might have missed. We don't have that. The teams are small and self-contained.

In short, we don't believe in TDD, but we would like to Unit Test. We just don't have the experience to do so, and we can't find it easily.

Answer 21

I know one person who will look at a form with a single button and say "That's overkill, this should have a command line interface, so it can be automated in a script" and yet he never really got behind unit tests. I just don't get it. Fear of change could be part of it, but I've offered to walk him through the process and he's declined.

Obviously my current company doesn't enforce them, I've just taken it upon myself to write them when possible. I have found that setting up tests up front leads to witting better/cleaner code, because in thinking how to test something you generally have to consider it's later usage.

Answer 22

Of course, in the ideal world, you can not argue against having a unit test.

However, whether you write a unit test depends on a number of things:

• How the software will be used. If you were writing software for just yourself would you write unit tests? Probably not. If you were writing pre-packaged software to be sold commercially, probably yes.

• How many people maintain the code....if it's just you, then you may know it well enough to be confident enough after making a change that a quick run through of the code is sufficient to ensure nothing has broken. If other people who did not originally write the code must now maintain it then a unit test will help give them confidence that when they update the code to fix a big (that was obviously not captured in the unit test!) they have not broken anything.

• code complexity: only test code which needs a test. A one line variable assignment method does not need testing. A 50 line method with multiple paths of execution probably does.

• Practical commercial commercial considerations: The fact is writing unit tests does take longer than not doing so. If you are writing prototype software, that has an uncertain commercial future, then there is a pay off to be has between having code quickly, now, that works sufficiently well versus having unit tested code in 2 weeks that works better. Sometimes it pays to find out quickly (consumer appetite) if software will have a short shelf like and move on to the next project.

and as others have pointed out, a test is only as good as the person that wrote it.

Answer 23

I've worked for 3 companies so far, and have just recently started writing "offical" NUnit tests, and I have to say I'm a big fan. There are 2 points I've noticed however in the last few weeks:

1. It depends on the application - there are certain apps/pieces of an app that can be very easy to unit test. A function that does something basically the same each time, and produces an easily recognizable result. And there are sections that don't do this: a block of code that prints crystal reports or modifies an image - here the eyeball is actually needed and probably better/faster. These areas can discourage people from trying to write tests. They then just start thinking that if certain areas can't be unit tested, why bother?

2. The granularity of the test- many DEVs/QAs already have some sort of automation in place to 'test' a piece of functionality, and get confused as to how this is different from a unit test. Depending on their test, it might not be. The important thing here that takes some time to appreciate is that a good unit test is totally granular. It tests the minimum piece of logic that makes sense, which makes it repeatable, automatic, and hopefully always valid. It took me a little bit to really appreciate how useful that is, especially when you have a huge codebase and like to run regression tests after making a change to see if you just foobared the whole app.

Answer 24

I think it's worthwhile to write a good unit test coverage for the code that you gonna be working with way into the future. For example, when you are working for a product company, by writing quality unit test cases you are simply making a time investment, knowing that all the hard work writing test cases will pay off and make your life easier down the line.

The sad truth is that a lot of software programmers work on in-house application development or contracting and just trying to get some software out the door. In those cases, the time investment does not make that much sense, as chances are you never see this pile of code ever again.

We can all argue that adhering to solid software development practices is responsibility of every developer. However, ROI means a lot to individual developers as it does to management.

Answer 25

This is going to sound a little like an AA meeting.

I am a software engineer.

I was talking about software engineering to my brother at XMAS (he is a business analyst).

• He has been working as a BA for 10 years.
• The conversation got around to testing.
• He had NEVER seen (or heard of) automatic unit testing.
• Or even heard of developers doing it.
• In that time he was worked for some of the big "alphabet" companies, on large government contracts.

So as far as I could tell, of the 100's of developers involved, none of them had heard of unit testing.

My predecessors in my current job used automatic validation tests, and had very few unit tests. Needless to say the code needs heavy refactoring.

Answer 26

If you want to sell everyone on testing do the following:

1. Write a bunch of tests.
2. Notify the other developers who change code and fail your test(s).
3. They'll fix their code.
4. Now you can release without those particular bugs.

Even a manager could understand this.

Answer 27

I don't think laziness is the root cause of bad unit testing. For my company, time constraints and "just get it done" attitude are the biggest deterrents for doing unit testing. Also, the places where our systems fail tend to be more at the integration level (services, database accesses, complex queries that require specific data for testing), not the "unit level." These things are just harder to test, and if you barely have enough time to get the feature done, you're probably not going to have time to get any useful tests done at the same time.

Answer 28

My 2 cents:

• It requires some education and discipline, but new graduates already come with the proper knowledge.
• Tests overhead can be reduced with better tools and this is happening also (refactoring etc.)

So, it's just a matter of time.

There's the Martin-Coplien debate in which Bob Martin asserts that:

"nowadays it is irresponsible for a developer to ship a line of code he has not executed in a unit test."

[http://www.infoq.com/interviews/coplien-martin-tdd]

Answer 29

Unit testing is one of those black box terms that most people have heard, but don't know what exactly constitutes a unit test, where to start, how to write them, how to actually run tests, what exactly they should be testing, etc. etc. etc.

In a lot of cases, its easier for the unsure developer to simply dismiss them as unnecessary or some icing that only "enterprise level developers" need.

Answer 30

The unit testing is great (easy to implement, easy to determinate that you have wrote enough test examples) when you have some code that operates as a server.

Example 1: You should mark in database posts that have 3 or more site moderator reviews What are tests variants

• check post without comments -> expect no marks
• check post with one comment -> expect no marks
• check post with 3 comments by common users -> expect no marks
• check post with 3 comments by common users and at least 1 comment by moderator -> expect no marks
• check post with 3 comments by common users and at least 3 comments by moderator -> expect mark for this post

explicit?

Example 2: You have a web site page that output the list of posts

How we should check that this list is misaligned in some browser? How we should check that this list is colored properly and we didn't crush styles?

there is no easy solution to automate this

so the task during unit test creation is to understand - can you put code in server operation mode - if you can create test, if not then break to pieces and create test for the pieces or test manually.

And one more thing - the economy of unit testing you will get if you have more than 10-12 testing stages without significant code modification.

Regards, Pavel

Answer 31

I think that the programmer has to just start doing it. A few simple tests to start with are easy to justify as part of development.

Something like a unit test is almost always necessary to get fast debugging turn around. Just explain how much faster it is to launch the test than it is to arrange the correct input, set a debugger breakpoint, launch the application, etc.

Document the test in your code. Just put a comment explaining where the test is and how to run it. Future programmers will see it and hopefully the testing will spread!

Answer 32

Companies are not performing unit testing, for the same reason that many websites are written poorly – ignorance, and people sticking to old habits. At my company, since we started unit testing (with Nunit, and Typemock), we reach higher code coverage and release the software in a shorter time to market.

Answer 33

The main reason is that many developers and development managers don't have a clue that unit tests exist, or how to use them.

The second reason is that unit tests can only be used (in a sensible manner) with code that already fulfills some quality standards. Chances are, that some existing codebase doesn't fall into that category.

The third reason is lazyness and/or cheapness.

Answer 34

I think the main reason is simply that most managers and developers just don't care enough about quality. So they tend to believe that writing and automating unit tests is just too much effort (which it actually is) for little benefit.

Maybe there is not much we can do about the benefits, but we should be able to reduce the costs of creating and running tests by improving the relevant tools.

I find most current open source tools for developer testing to be quite primitive. Unfortunately, there doesn't seem to exist much interest in the industry to improve this situation.

Answer 35

Because unit tests are only usefull if you write testable code. And writing testable code is hard. And people are lazy.