Give short concrete answer: why unit testing did NOT work out for you (your project). Will you particularly try again on a different project?

+47  A: 
  • Because my project was big and already started, and it was hard to build up the base of unit tests to make the effort really worthwhile.

  • Because the architecture of ASP.NET WebForms/[your platform] makes applications difficult to test.

  • Because not everyone on the team understood the testing strategy/knew how unit tests are supposed to work.

Dave Markle
"Because the architecture of ASP.NET Forms"... You can test all your Model and Controller... you just do not test your Visual... This reason is fake sorry. About 80% of your application should be as easy to test than other technology ;)
True if you're using ASP.NET MVC, not so true with most usage patterns of ASP.NET Forms.
Dave Markle
"Because the architecture of ASP.NET WebForms/[your platform] makes applications difficult to test." Oh man, did I miss a terminology meeting? Please don't tell me that we're allowing what is done with most WebForms solutions to be called **architecture**
George Mauer
@George: Ha! Don't worry, I'm talking about the architecture of the platforms, not she solutions :)
Dave Markle
+74  A: 

It didn't work out for me on my project because I didn't use it. I didn't use it because I didn't understand it.

I picked up a book on it, and won't make the same mistake twice.

George Stocker
I'd also recommend the Pragmatic Unit Testing book from -- comes in C# and Java flavors. C# version is at
Thank you. I'll give it a looksee. Since I have a Safari Library subscription, it makes this a lot easier. Otherwise my home library would be chock full of very expensive books.
George Stocker
Same here, I didn't understand Unit Testing properly.
Good job getting around the question.
@MusiGenesis - I'm not sure why you voted down my answer (though it may not have been you, the comment and the recent downvote would point in that direction) but not understanding the problem domain is a valid reason for TDD not working. 23 other people think so, at least.
George Stocker
@Gortok: it was me, nothing personal. :) The votes here and for the next top answer are votes for unit testing, which is fine, but as I expected they're blocking the real answers to this question.
@Gortok: I block legitimate answers with jokes fairly often, so I shouldn't be talking anyway. :)
As I mentioned in the question comments, I really would like to see the problems people encounter with this addressed with something other than "you need to unit test". Testing code that talks to network resources that only exist in the production environment is a big question for me.
Also, saying unit testing didn't work for you because you didn't use it is like me saying Oracle didn't work for me because I used SQL Server. :)
Somewhat. Though those are comparable items, Not using Unit Testing vs. Unit Testing isn't comparable. Once you've been exposed to a system with no Unit tests, you start to clamor for a better way. Believe me, I'm there now.
George Stocker
I undid my downvote. Your ascension to the top is guaranteed, and you did vote for my answer. :)
You wont' make the same mistake of not unit testing? Or you won't make the same mistake of picking up a book on it?
Just kidding. I won't make the mistake of building code without unit testing next time.
George Stocker
I can't believe people are still voting on this question.
George Stocker
I can't believe this question was only a month ago. It feels like 5 years have gone by.
In the terms of the internet, they have. :-)
George Stocker
not understanding something is never a positive
+71  A: 

Undisciplined developers who stubbornly refuse to change working habits can make this not work.

So do you think your colleagues are stubborn and undisciplined? If it was that good - why wouldn't they just change their habbits themselves?
In small teams of maybe 4 people, 2 people against testing effectively blocks it. Fortunately I've long since found new projects :)
There are a number of reasons why people are undisciplined. Some think they are artists, some think they are geniuses have love problems, ADD, burnout, gambling, disinterest or a sick mum. Testing is the most subtle skill og all programming; it takes years to perfect - if ever
I would never dream of accusing a colleague of being undisciplined and stubborn ;) Most people only want to be shown HOW to be good at it.
+58  A: 

Missing separation of concerns. GUI intermingled with business logic code.

All on a project that was already started

This kiond of thing can make unit tests very difficult.
Another good reason to do TDD -- making things easy to test can drive better design practices.
TDD won't help if the project already miss seperation of concept. TDD work only on code that is well seperated. Unit testing code in the GUI never work well.
If you use TDD, it forces you to separate the concerns. Otherwise TDD will quickly grind to a halt, because making changes in bad code is hard. Then it forces you to refactor the code better. The GUI can be easily unit tested when its presentation is separated from its logic.
Esko Luontola
@Esko - the project was already started, as Tigraine says. That means there was already code where presentation was not separate from logic.
+7  A: 

Unit testing has failed for me in the past (or rather, I failed at unit testing) because I have not thought far enough ahead and not tied down the design early enough. By the time the system was well enough specified for unit testing to be beneficial, it was already too much work for the nature of the project to implement it.

THINK AHEAD! - that was my mistake.

Unit testing supports agile, which explicitly targets the old "think far ahead waterfall" model. The idea is to code something working and fully testable, based on what you know now. Good design practices help you weather the coming changes which are definitely coming.
+15  A: 

Reluctant colleagues - and the lack of persuasive ability to talk them round.

A manager needs to push it, otherwise it won't be maintained in the long term.
I disagree, if the developers doesn't see the point of TDD the managers can push as hard as they can and it still won't happen. The same is true for the opposite, if they find it helpful they'll use it regardless of management. The bottom line is what counts and that bottom line always is working software with high quality.
+10  A: 

A particularly annoying area of development that is really badly suited to unit testing is the growing trend for frameworks. Unless the framework - eg a MVC solution - comes with mocks, it is ludicrously difficult to unit test code developed to run on to of that framework. I generally just give up on unit testing in such circumstances.

David Arno
@David, I (and many others) share your frustration. However, it's not terribly difficult to create an interface and wrapper for classes that don't expose default constructors. ASP.NET MVC has done this for many of the hard to mock framework classes already making testing easier.
@tvanfosson, firstly please note that MVC != ASP.NET MVC. The latter is but one of a great many implementations of the MVC pattern. That said, as Phil Haack is both a keen fan of TDD and a key part of the ASP.NET MVC framework, it doesn't surprise me to learn that it is easy to unit test against. :)
David Arno
check out Rails... for how to "build in" testing into the framework. I'd say it's one of the best things out in the last 5 years.
+13  A: 

Started writing tests after the code was written and got frustrated because I felt like I was wasting my time. The solution was to switch to test-driven development where my unit tests drive my design. Now each test is progress instead of waste. I'll never go back to developing tests afterwards.

EDIT Now, even on projects that are fixes or upgrades, I still do TDD. I won't go back and put in tests for existing code, necessarily, but for new code or bug fixes I'll write at least enough tests to ensure that my changes don't break anything. Tests are written first to verify/protect the existing functionality, then to introduce new (fixed) behavior.

An upvote for developer reform.
+1 - nice answer!
+8  A: 

I employed unit testing in a couple of projects I worked on (web applications, using business objects + stored procedures to perform CRUD operations on SQL databases).

Heavy use of SQL prevented us of creating really fine-grained tests. To properly test some database-interaction method you need data in the database (that is, if you're intending to automate tests). And data can only be added via another method... so you're testing both.

I ended up either writing not-so-grained tests that verified all inter-related methods - or living with the fact that a single error brought down multiple tests (for those of you thinking about mocking: part of the bugs found during unit test were in SQL, so mocking was excluded on purpose and I'm glad we did it).

Dan C.
We don't use SQL directly where I work. Instead we use stored procedures. I get the fun task of writing my own SQL to insert test data, so I can turn around and test the stored procs/funcs, and THEN test the code that calls the stored procs/funcs.
R. Bemrose
With "heavy use of SQL" I meant database access, actually (see the first paragraph). And yes, there's also the possibility to write SQL just to insert test data, but it seemed overkill for me. Time to complete the projects was limited, you know... :)
Dan C.
Mock out the database!link in a library that defines all the functions your database access code calls. Once you're able to control what happens when the code calls db_open or db_exec, you can do anything.
Mock out the database? "You can do anything" isn't a good thing. The tests should be ensuring the sql operations are valid. If they require data then static/dynamic fixtures are in order. If you're going to mock out the database in a SQL heavy app, you might as well just mock out your assert statements too.
+36  A: 

Unreasonable deadlines that barely allow enough time to write the production code. I have used unit testing successfully on previous projects and will continue to do so whenever there is enough time. I guess it's a matter of helping management and the customers to see the green bars at the end of the tunnel.

Up voted for "green bars at the end of the tunnel". Haven't heard that one before. Excellent.
Paul Batum
I’m curious. I am a unit testing skeptic, but I though one of the selling points of unit testing was it made development easier (i.e less bugs, less re-writes, less re-design). If that is the case shouldn't that mean development is faster so you need less time to write the real code?
I've seen this false economy in practice numerous times. Testing (any testing, not just unit-testing) feels like "unwork" to many PHBs and so it gets bumped in favour of just "getting the job done". Of course, now you probably have to get the job done twice. Once to build, once to fix after launch.
Toby Hede
With a reasonable amount of good unit tests, the total time to deliver a high-quality product is lower than without them. But it takes less time still to deliver a buggy, barely functional product if you cut corners and don't do any tests.
Michael Borgwardt
@jussij - I think it depends on the developer. If the developer really "understands" the business requirements, unit tests are often a waste of time. Unit tests approximately triple your lines of code, which is a lot of overhead to overcome to make them more productive. Plus, they don't make much sense to business users, who really care about integration tests.
Unit testing is an investment that pays off after time. When you've got test coverage you've got the safety net that will prevent you from stumbling and falling down later on during the development. It's easy to see the bugs that's found in production but much harder to count the ones caught by unit tests during development...
I once had a discussion with a manager that stated that unit tests slowed us down. My proposal was to use method, variable, type names with just one or two letters to increase typing speed. He saw the light after that proposal.
Pedro Santos
+34  A: 

Ego, most team members felt their code is superior enough to not require Unit Tests, at the end of the day it's a cultural change.

Student for Life
+1 "cultural change" in bold
Good is the enemy of great
+55  A: 

Tightly coupled code made it impossible to test one unit without testing the entire system - which in turn depended on other systems.

Nils-Petter Nilsen
common excuse from what I hear
That's something that we struggle with quite a bit as well.
Thomas Schultz
The book "Working Effectively with Legacy Code" can help.
+2  A: 

Unit testing did work for me and made my life easier on my last project.

But because my object model has crud functionality built right into the objects, testing against a database was an unbelievable PITA! Next time, the crud's going to be separate, and I've got to look into Mocks.

John MacIntyre
Did I understand you correct, your domain classes had functionality to save, delete, update and so on?
Yes. It worked ok, but there were enough problems with it to change it in the future.
John MacIntyre
It's the Active Record pattern. Works great for smaller projects.
+5  A: 

it is very hard to write unit testing for low level hardware development.
How do you unit tests your interrupt handler without writing full dma logic that will trigger that interrupt ? Actually we are doing unit testing just the units are quit big :)

why can't you test the method itself, separately from the triggering? And just "trigger" it from the unit test?
because it will be useless, interrupt service routine should process real data coming from hardware, simulating hw is theoretically possible, but practically not applicable.
Create a function/method that does only the hardware independend data processing. The data and everythings else are parameters to this function and can be faked/mocked...
unit testing is not a religion it's a method of reduce debugging time. if to achieve this you need spend twice time that saved in the debugging ... do you math .
Actually, your answer sounds like it would make a great question for StackOverflow.
Tim Lesher
I don't like this kind of questions. It's too open. This site supposed to serv simple question/simple answer approach and not for theoretical discussion. But be my guest :) If you want to ask i will not claim for copyrights :)
+3  A: 

It didn't work for me in a project where we had shared code ownership (usually a good thing!) and I was the only one writing and executing unit tests.

I tried it in an existing project with highly coupled classes and not much cohesion. Unit testing only worked for me here in new pieces of code.

The first project I tried to do unit testing things went sour because I didn't use TDD. I added all unit tests later and lots of times didn't have time left to add the tests or had to rewrite lots of code to be able to test small portions of it. Unit testing cost more time here than it saved although the few tests we had did add to the quality of the project.

In short you have to have support from coworkers. You have to have a codebase that supports unit testing (or you can add it in by refactoring the old code-base piece by piece) and you have to do work test first at least when figuring out api's and design. It also helps to understand that unit-testing is more of a design activity than a testing activity.

+8  A: 

I've never used it. I've never felt I needed it, and it looks like far more of a PITA than just dealing with the problem it's supposed to solve. I refuse to take the time to learn and evaluate it in detail because of this instinct - I'm prioritizing things that I think will be useful.

Now, before you advocates explode, please feel free to explain to me why I'm wrong in the comments to this answer (as politely as you can manage, please - my position is not set in stone, and I'm just trying to be bluntly honest above). Give me a reason to love the object of your affections, and not an "and the advocates are all code-religious idiots" addition to my list :)

Josephine, I shared your views. What swung me was actually seeing the result from my first TDD experience. I think you tend to make a lot better design decisions that way. That is what sold me. Now, in later being able to go back and modify that code safely since it is all tested is a huge bonus
Testing allows you to forget about a piece of code you've written and concentrate on the next piece. Without the test, you can never be sure that the next feature won't break something you already have.
Aaron Digulla
@Aaron: sound design and the compiler also give you this security. Less security, true, but at a lower cost than maintaining a test suite (which is really another application, hence greater complexity and development time).
Alex, Aaron: Thank you for your replies. Without wishing to be arrogant or patronising, design decisions are my strong point. I don't have a problem with breaking old code as I'm ultra-strict with my coding rules, (refined for reliability over years). My team hate me for it, but it works :)
Josephine, assuming you don't have a problem with breaking old code, what about the other developers that will maintain your code? Wouldn't they be better off if a red light came on whenever they managed to break your code with their "not-so-strict coding rules"?
Paul Batum
+1 MusiGenesis. There are pros, but not nearly enough to justify the cons in my case.Paul: I read Josephine's comment to mean 'her rules' as in 'the rules they must follow', hence the hating.
unit testing provides three things of benefit that IMHO outweigh the perceived 'overhead': 1. a safety net for changes, 2. a sense of TEAM confidence in the code base, 3. how-to documentation that cannot get out of date (without failing).
Steven A. Lowe
and three more things: 4. a place for regression tests, 5. fodder for demos, 6. the ability to test the system fundamentals automatically, instead of manually. The latter saves many man-hours, which translates to money and productivity. This alone can often more than make up for the 'overhead'.
Steven A. Lowe
1. The compiler doesn't detect runtime error conditions, and even sound designs can have errors.2. I consider myself a good designer, but seeing the object in use can be more effective than just "imagining" it in use.3. Executable documentation for downstream or maintenance developers.
Tom Barta
Josephine, if you're a successful developer, I'm sure you *do* already unit test. Maybe you don't automate it, or more likely you rolled your own automation. The "refuse to take the time to learn it" comment suggests that you think it's more than it is. These unit testing frameworks are very simple.
Josephine, I'm with you on this one. As MusiGenesis suggests if you spend time getting the design right, more often than not everything else will take care of itself.
How do you catch value errors and, more importantly, ensure that your application's error-detection code is functioning? Is this via some form of broader testing and the only thing you are opposed to is unit testing? I write tests as mental notes, as I'm doing the design and thinking of edge cases.
Andy Dent
I like unit tests, I hate TDD. I can't believe people advocate refactoring over and over. Smart designers can come up with a solid design for even complex problems before starting, and then choose to test as they go or at the tend.
Good point LuckyLindy. I'm not sold on TDD either...
This is also interesting but doesn't answer the question.
Arne Evertsson
In the short, I'm torn on Unit tests. I've only written code for my personal projects, and I have yet to actually ever break a test that wasn't caused by a Hibernate or Maven bug. That said, I'd rather spend my time on Integration testing. I haven't dealt with Mocks yet; testing J2EE apps on JBoss, I just haven't gotten around to figuring out, but I think that offers better opportunity to test defensively.
Chris Kaminski
I don't think TDD == "refactoring over and over". Start with something that works and solves a simple test case. When a new issue or use case appears, and you find yourself in the same code, evaluate if a refactoring would simplify. If so; do it. If not; don't! Then, re-run ALL tests. Simple, ehh?
+20  A: 

If you're adding unit tests to an old "working" system, you'll face these issues:

  1. To make the system testable, you'll have to change things that already "work". Some of your changes will break the system and if the team hasn't bought the testing concept, yet, they'll blame it on the "testing hype".

  2. Your tests will show existing issues in the system which were gleefully ignored in the past. Nobody likes it when you find bodies in their backyard and guess who will be blamed? The long standing member of the team (who wrote the bug) or the new guy?

  3. In all projects, time is scarce. It's by design: If there was spare time, your boss would find you something productive to do. If developers don't get the feeling that the tests help them to meet their goals (by reducing unnecessary bug hunting and by greatly improving self-confidence and code quality), they will quickly stop doing this "futile crap".

Aaron Digulla
+30  A: 

Lots of unit tests were written, but they were not maintained vigorously enough.

All unit tests passed at the time when they were written and checked in.

The unit tests were run frequently, but when some of them started failing (often because the code changed without updating the corresponding unit test) we did not fix the unit tests quickly enough.

The situation was allowed to deteriorate and now we have so many unit tests failing that they have become almost meaningless.

Running a unit test suite before a check in to make sure I am not about to break anything is useless because lots of tests are failing with and without my changes, so it is impossible to see which failures were introduced by my changes.

Cayle Spandon
unit testing requires self-discipline. no self-discipline, no benefits
Steven A. Lowe
"now we have so many unit tests failing that they have become almost meaningless" -- I love this statement, brilliant.
lubos hasko
you should write the tests before you write your changes. if they break its your job to fix them on the spot. if its someone elses fault go up to them and scream. Works for me.
"now we have so many unit tests failing" is sort of like doing html validation at After 300 errors you just give up and keep coding the same html3.2+ crap with 12 level nested tables and font tags wrapping a single nbsp.
Christopher Mahan
+40  A: 

We confused Unit tests and Functional tests. Half are functional, half are real unit tests. Thus, our unit tests take WAY too long to run (over an hour). This makes our tests much less effective.

Brian Genisio
I've seen this fixed after the fact by adding a macro to have the slow, functional tests only built/run conditionally
+2  A: 

One of the systems I work with is a COTS (Commercial Off The Shelf) package with an extensive DSL (Domain Specific Language) used for automation (think VBA, and you won't be far off).

The vendor provides no support for unit testing, and the language has no inheritance, no reflection and no variables with local scope. It does allow for file inclusion, but each program (at least it supports multiple programs) can only have one "start" method in it.

There are only really three ways to implement any kind of unit testing:

  • Build a simulator emulating the core software.
  • Manually construct scripts that test each important function, accessed through the file include mechanism, and build some way of reconciling results
  • Build a code generation tool using comment-based annotation processing to automatically create scripts to run any annotated test methods and reconcile results.

In all cases, once we created our framework to make testing possible, we'd then need to start creating tests - and figuring out a way to test anything, without trampling over live data or having dependencies on other tests, will be extremely, how shall we say, interesting.

All three methods require an enormous investment of effort, and we have so far been unable to justify making the large investment of time.

Besides, this system will get turned off 'real soon now'!

Bill Michell
+128  A: 

Unit testing did not work out for me on one project because the other developers started to act as if the test suite was the real application deserving of their attention and love. They spent most of their time on the tests, and the actual application slipped well past the deadlines. As a note of caution re: unit tests, when a customer is literally yelling at you because an application is not done, it is inadvisable for a junior developer to blurt out "but we've written 5 times as much test code as actual code". Whether you use unit tests or not, you have to admit that it can be a tough sell to the customer.

Also, the unit testing in this case was largely a failure because the tests were mostly aimed at aspects of the application that were never going to change, and ignored the weakest points. In this case, the weak points were the availability of network resources and the state of the data in the production environment. All of this was mocked in the test suite, and essentially useless. This incredibly heavily unit-tested application failed miserably in the field.

So how do you balance the need to unit test with the need to get an application out the door on time?
George Stocker
I voted this answer up, well thought out.
George Stocker
I've thought about this too, off and on. At least in a lot of business-oriented development, the real key is those integration tests. It's easy to forget and act surprised when your app with hundreds of unit tests and 100% coverage crashes hard.
Nicholas Piasecki
@Gortok: "unit test + app" should take less time than mere "app". There is no balance here.
J.F. Sebastian
@J.F. Sebastian: You're absolutely right, but if you don't start an app with unit testing in mind, or you're dealing with a legacy app with no unit tests or documentation, then it would be harder. See:
George Stocker
@J.F. Sebastian No, there most certainly is a balance. Like all good things, unit testing CAN be overdone. You eventually reach a point of diminishing and even negative returns, where you have too many tests that do little for app quality but require significant work to maintain.
Michael Borgwardt
@brazzy: this is my whole issue with unit testing. TDD advocates tend to not acknowledge that automated tests represent an increase in the overall complexity of a project, which all else being equal is something to avoid.
This doesn't seem like a failure of unit testing, this seems like a failure of management. Was there nobody tracking the progress vs. the schedule? Did management not know where time was being spent? Was there no ability to course correct? The tests seem like the proximate, not ultimate, cause here.
Jeffrey Fredrick
@Jeffrey: agreed, but you highlight my main point. A programmer can say "I use unit testing" and still be completely incompetent. I can (and do) say "I rarely if ever use unit testing" and be perceived as incompetent.
I have worked on many VERY large enterprise apps, and for anything other than APIs I've never seen Unit Tests work. Integration and QA tests, yes, but never developer-specific unit tests.
Unit testing doesn't test everything, and isn't a guarantee that it'll work in the field. If you put all of your testing faith into just JUnit, its like only wearing one shoe out the door.
Dean J
@Dean J: I would say it's like wearing one shoe on your head.
It's a lot to do with coverage vs maintainability vs effort cost. Anyone who tells you unit testing has no effort cost is lying to themselves. All code comes with a degree of maintenance, and when the team stops fixing broken tests, unit testing loses its advantages - and fast. There is a comfortable middle ground, finding it can be challenging.
+2  A: 

When I first tried it way back when, when NUnit was pretty new, it was difficult to handle testing of UI elements in ASP.NET and to mock up 3rd party tools or applicatios (Outlook, etc.) that the app needed to talk to. I suspect now with better mocking tools and UI testing that it would be easier, but at the same time, going through the effort of setting up such mocking frameworks does take some additional time.

I'm not saying that it's not worth it, the last project I was on where we used them, we had them all hooked up to CruiseControl.NET and it was awesome to see all the builds and tests run on each checkin....

Sam Schutte
+15  A: 

TDD has never made sense to me because:

  1. Who tests all this test code you write?
  2. Vast majority of defects are UI related and won't be found by unit tests
  3. Rest of errors are due to missing functionality that if you forgot to write you would also forget to test (eg: oops, no validation or oops sql injection)
  4. Every change now requires changing the real code and some test code - double work
  5. Impossible to unit test a complex system without leaving a certain amount of artifacts in the production code to support testing
  6. Provide a false sense of security to many people because they think the codebase can't be messed up because there are tests.
  7. I think TDD grew up with dynamic languages and is the replacement for not having compile time checking
  8. Ive never had a unit test catch anything but the most simplistic error that would have been caught anyways if the developer had bothered to even run the code.
You can't write code to test the tests because if you wrote code to test the tests you'd have to write code to test the tests that test the tests. I'm going to delete this in a second, but that was fun to type.
Actually we are drastically increased a number of automatic tests in our last version, which is currently delayed already by 2 months because of bugs in the test code ... (not counting the time spend on writing tests). Time will tell if we will return the investment in next version releases.
Actually, unit tests have been around since assembler language, and it was quite common to have unit tests in C using #ifdef MAIN or something like it. What's new is things like junit. The best way to think about unit tests is as a kind of spec: what must be true for the code to be "working".
Charlie Martin
1. The test and its subject are tested simultaneously each time you run it3. The act of writing tests helps identify missing functionality4. The intent is that automating the tests is *less* work than the manual testing you would need to do instead. If this isn't true, then don't unit test.
6. Those people need to be educated.7. Possibly so. I like having both though.8. One reason you haven't had unit tests catch significant bugs *might* be that you don't buy the approach, and therefore haven't written many tests.
@MusiGenesis: It looks like in your world, seconds are incredibly long.
+1 A very nice list.
I agree with some of this, but #8 is surprising, as I've found some very tricky logic bugs thanks to unit testing. Also, writing tests tends to generate code which is simpler to understand, this counters points #4 and #5. Personally, I'd rather work on a product with good tests than one without.
-1 most of this is flat out wrong (2) or betrays a total misunderstanding of what constitutes useful unit tests (4, the point is to test wanted functionality, not implementation details).
Michael Borgwardt
-11 you? your peers? wtf.2 Unit testing is mainly there to remove glaring coding mistakes and bugs. UI's can be tested by other means. You don't understand this topic.3 You are right, you can only test for things you know might break. Isn't that the point? How would you test them otherwise? Unit tests aren't there so you can be one that knows nothing.
4 Its a trade off, you do a little extra now, but have to do WAY less debugging/tracing/maintenance in the future. Cutting down on 90% of my job would be amazing.5 If its not testable you have a design problem. Somethings aren't testable but they are few and far between.6 Its not TDD's fault you don't understand it and are therefore scared of it.7 I think you don't think.
8 90% of my work is involved in fixing these 'stupid bugs' that anyone might notice if they looked. People don't look though (or simply miss it), thats the point. It also allows you to test integration changes that may break other parts of the system. This probably wasn't seen by you as it sounds like you didn't use it for very long.
Interesting answer, but it is not a short concrete answer about personal experience with unit testing.
Arne Evertsson
Like @Si, I'm surprised by #8. Aren't unit tests a way of running the code? And surely a better way of running the code than haveing a developer do it manually.
-50 If I could. Seriously, how can this get upvoted? I mean, WTF, everything in there is so wrong. I could answer all points one by one but I feel like I'd be loosing my time. Happy manual testing! (Luckily, I'm not working with you, at least I hope so).
Pascal Thivent
Guess I'll pile on. First I assume you wrote this to get a reaction. If not please reconsider your phrasing as it is provacative.#1. The assumption is that you will rarely make a mirror mistake in the test code as a mistake you made in the real code. This is true, but that does not mean that you did not make a mistake in the test code that does not fail, yet. I have run into situations where the real code was changed to correct an issue, and the test code began to fail because it had a different error.
Pat O
#2 Defects in UI should be limited to visual aspects if you have proper code seperation.#3 Granted, except if you do TDD. With TDD you first render the specification as a set of tests, then write the code. The point of TDD is to keep you from forgetting some function.#4 Granted#5 Depends on the architecture. If your code cannot be seperated into units, then it cannot be unit tested.#6 True, but then so does any number of other methodologies.#7 I have used dynamic languages with and without unit testing. I do not find that I "miss" type checking either way.
Pat O
#8 I find that unit tests help me catch things while I write them. Rarely do they fail once I am done writing them (except when I am using TDD). Instead I find that the act of writing the unit tests forces me to look at the code in a different way. The result is better code. Perhaps you are just a better programmer than the rest of us. You can think as a developer and user of your code simultaneously. You rarely make bone headed mistakes. That is great, but I am not like that. After 20+ years of software developement I will still take all the help I can get.
Pat O
-1 you aren't getting the whole idea behind unit testing and what a unit test is.
-1 either troll or a Willy
just somebody
This entire list is more or less how I feel after a month of unit testing. #7 really enlightened me.
+1  A: 

1) Legacy code. 2) Strong dependency on a framework (WebForms for example) that don't enable easy testing - too much work and workarounds that clutter the code just to gain testability.

+4  A: 

Unit testing did not work for me (a few years ago) because the code I wrote was not designed to be tested. Lesson learned: take the time to hone your code like a fine piece of woodwork.

Andy Gherna
+35  A: 

Unit testing isn't a silver bullet.

Our product at work is a 40 kloc (Python) application with ~120 kloc of tests, and the full test suite (including functional tests that remote control the GUI) take hours to run (distributed on several integration machines).

We have some testing problems:

  1. They take too long to run - we often just run the unit tests to check in, which occasionally leads to broken integration builds and backing out commits.

  2. Some functional tests fail spuriously. Sometimes this is due to weird screenshot failures on Vista, sometimes tests are sensitive to timing differences. When we see these we add retries or try to make the interactions more deterministic, but chasing these false alarms can cost a bit.

  3. Some unit tests are written in a very 'mock-heavy' style - they know too much about the implementation, so that when it changes you have to change the tests too, which can be a real pain. It's tricky to find a balance between mocking out too much stuff on one hand and having tests that test too many layers on the other, but we're still learning.

However, they're definitely a net win. When we fix defects, we add tests to avoid regressions (as well as the tests we create as we build features), so changes to a complex application tend not to break other things.

That gives us a huge safety net, of the kind you couldn't get from compiler checks (well, the Haskell/OCaml people might say otherwise, but you certainly couldn't get it from a Java compiler).

We can make big changes and be confident that we haven't broken things. I find it hard to imagine working on a largish application without the security a large, well-maintained test suite gives. Well, working effectively on one.

It doesn't save you from bugs - we (or our users!) still find things we didn't anticipate that break. But it helps.

So you have a bad test suite - one that is slow, brittle and too tightly coupled with the production code - not FIRST ( It's good to hear that even even though you have problems with it, you get value from it. An ugly test is better than no test ( As with anything, doing things well requires skill, and making a good test suite requires skill.
Esko Luontola
With that many tests, how do you get around the "let's turn off the broken tests" (rather than fixing them) problem?
Discipline, basically. Also, the manager understands the value of the tests, so there's more pressure to get the test working than to turn it off.

I have a client whose codebase isn't well integrated into the IDE (eclipse) and so the code is essentially uncompilable on the desktop. I can't get the unit tests I write to work without major work, which I can't justify yet. So I work on fixing the project build path slowly, when I get the time.

+3  A: 

I know you are looking for complete failures, but I have found that unit tests are not enough though they certainly help. I work in moderately large systems: 1 million lines of code, multiple services per server, around 100 servers per cluster. We have thousands of unit tests, but those cannot capture the interactions between all these components. So in addition, we have a few thousand integration tests that run the complete system against pre-defined scenarios. In addition, we run new versions of the system in parallel with production against carbon copies of the live data. And despite all that, we still have problems a few times each year.

In another really tiny project (2Kloc) I wrote 3 times as much test code as actual code. I even pushed the code coverage to 100% (at least according to gcov,) yet we still found a few bugs. That might be considered a success (if the defect ratio is acceptable to you,) or a failure (if you think that code coverage and unit testing is enough.)

"Testing shows the presence, not the absence of bugs." --Edsger W. Dijkstra
I completely agree. You want layered tests - unit tests, integration tests, acceptance tests, and even live human testers if possible.
+3  A: 

Because some programmers on projects I've been on didn't understand Unit Testing or how implement it, and once shown, had the hand-slap-forehead moment. Also, the tool didn't really provide the greatest support. Not everyone can afford VSTS.

Also, I used to write console applications to test my code, until someone pointed out what I was doing, then I realized I'd been doing it all along.

For many, TDD comes across as backwards, until you show them the "hello world" of TDD with Unit Testing, light bulbs go on, red ropes part, and life is good.

everyone can afford Hudson, CruiseControl or Maven though..

I think the two biggest reasons for the absolute failure of our unit test environment on my team were:

1) No permission to add the unit tests as a post-build step for each library, ergo nobody ever ran them after the initial implementation, ergo they no longer compiled after a few months.

2) People mistaking the unit tests for test apps with all kinds of database connections, file streaming, etc., leading to extremely long running times that encouraged everyone to exclude everyone else's tests except their own from the executables.

+17  A: 

About 18 months ago, I tried to add unit tests to a large pre-existing project, using ruby's Test/Unit (it's basically exactly the same as JUnit and NUnit). I'd used NUnit quite a lot before, but had never seen much benefit out of it, and was pretty jaded about the whole unit test/TDD thing.

I ended up with a bunch of 'shallow' tests, which just checked that the software did what it was doing, not that it did what it was supposed to do, and were very tightly coupled to the code.

After about a month or 2, these became useless and unworkable, and I deleted them all. My checkin comment was "EPIC TEST FAIL" or something like that.

About 12 months ago, I built a large new section of a website, but this time, wrote the unit tests as I went along, and used rspec. It's worked out as great success, having caught many subtle and annoying bugs, and it hasn't gotten in the way of refactoring the code. I attribute this to 2 things

  • Writing tests at the same time as the code, not trying to shoehorn them on later
  • Learning to focus on specifying desired behaviour, rather than asserting what already happened. I will forever be in debt to rspec for having taught me this. Go rspec!
Orion Edwards
+3  A: 

Because the project didn't use unit testing from the start. Now it has grown so huge, that it becomes painful to write unit tests for the existing codebase.

Yes it works out for new stuffs (I've tried it and loved it), but the lack of enthusiasm of my seniors to enforce unit testing has made my effort becomes useless.


Been there done that
+3  A: 

In our case the development team was relatively new to the notion of refactoring as well. Hence, it was impossible to keep the tests clean and understandable. Instead of having duplicated code only in the application we had complex, duplicated and intertangled code in the tests as well.

If you don't keep your test code nice and DRY it may grow into a big ball of bad-smelling mud. This will probably only slow the project down instead of giving the team increased development velocity.

Mattias Holmqvist
+5  A: 

I didn't work out for me until I separated my project code into libraries. After that, it has been super-successful, because I've had time to maintain it. I currently have about 1100 tests that run in 4-6 seconds.

As it is now, if I add new functions to this library, I start by writing the tests -- because this allows me to decide how the functions should act, then I write the functions to suit these tests.

Also, for functions that e.g. converts a phone number to a valid MSISDN number, if I change it, I can run the tests and see if anything depends on the old behaviour -- see if they break existing dependencies and expecations of behaviour.

The only part that could be better is the database interface. It does all kinds of tests, but some it can't run (like sending voice and sms messages). If these were to work we need to set up a 100% up-to-date test system. But it would be possible.

Absolutely great for testing against code that has a long lifetime.

+2  A: 

The most common reason it doesn't work is because the unit testing code is treated as a third class citizen.

That is the code is absolutely not written to be maintainable which quickly becomes a ball and chain. As more and more unit tests are added, older ones become obsolete and aren't fixed. Which means you end up with a mess of test code that's virtually useless and doesn't really test anything. But there's lots of it!

The key to success is to treat your unit testing code with the same respect as your production. Not any more, not any less, just the same.

As well, always decouple the testing code from the production code. As soon as you start adding testing code in your production code (rather than in your testing code), then you're asking for trouble...

Stephane Grenier
@Stephane Grenier: you might have a strong opinion on whether or not it is okay to have your tests production assembly (project)?
I'm sorry, I don't fully understand what you're asking by your "tests production assembly (project)"...
Stephane Grenier
+2  A: 

On the one hand, the project succeeded - there were almost no code related failures. On the other hand, the build took almost four hours to run and there were thousands of tests, Selenium and JWebUnit based, which were incredibly difficult to fix when they broke. Why? Because clever and talented developers had seen redundancy and replication, and had built clever abstraction layers over the top of the tests, which had the unnoticed effect (until years later) of completely enshrining framework and data dependencies. We eventually tried to move to SeleniumGrid and it was almost impossible to parallelize the tests, as the interdepencies went so deep. (We 'fixed' the problem by buying a much more powerful build server. Let's hope Moore's law keeps up with our test proliferation).

So I will try again on the next project. But next to DRY and YAGNI and all those good old commandments, I'm putting up TORT:

Test Oughta Repeat Themselves.

Oh, and before anyone jumps in and says 'Selenium is FUNCTIONAL, not unit testing', we found that once the testing bug had spread we were testing UI from a browser with completely mocked out Controllers. Caught a lot of navigational and AJAX problems that way.

Or at least, RYE: Repeat Yourself Enough. Readability is the most important attribute of a test.
Sean Reilly

when I'm programming some algorithmic or mathematic stuff which is hard enough to design. I create my test first which force me to think "what I want to do" ? Then I create my solution, so I'm sure my job is done when my test pass. I no longer need to create a console application, or step in my code through debugger(or juste a little).

Unless my code is published (ie is a framework for public use), if the problem is trivial, I do not make unit test (if you are a good developper your code must be always trivial).

Sometimes, I do some test on my interfaces, so classes which inherit my interface have automatically some test to see if it's well implemented. (for example, if you implement IList on your class, make sure that any not null Add, increase Count by 1)

But I think unit testing are a really good documentation (I've learn many thing in the Unity framework by looking their test suite).

I see unit testing like an extension of compiler errors.

Nicolas Dorier
you just reminded me of the advice I took from Writing Solid Code - I used to step through each line of code in the debugger, keeping an eye on variable values. Now I write unit tests. They have the virtue of leaving behind a regression suite.
Andy Dent

Because the developers did not understand requirements and specification of the business logic well enough to design correct test cases. It was especially hard for the developers to determine corner cases such as expected behaviors that make sense in business around the boundary condition. So most efforts have been spent on passing the specified threshold of the coverage.

+1  A: 

So far, Unit Testing did not work out for my Projects on Microsoft Sharepoint. The issue is that essentially I am doing integration testing (I have no experience with mocking Frameworks, and mocking SPWeb and all it's dependencies looks like a behemoth).

If you want a nice little bullet point:

  • too many additional dependencies

I still want to do proper Testing, be it Unit or Integration testing, but I really have to schedule some time for this.

Michael Stum
+1  A: 

I love unit-testing. It has saved me a lot of trouble.

But there are some situation where it doesn't apply, and in my case was when I wrote a relatively complex driver.

For simpler driver, I could create a mock object, which simulate the device's response.

But there are some devices, eg. Network Processor (NP), where it doesn't work well.

Of course we can simulate the NP, but the higher the layer you simulate it, the less effective the unit-test will be.

Some NP has a pretty good simulation, but I would call it an integration test, rather than unit-test, since there are so many layer of abstraction before I could test a specific object.


Every programmer has to ask themseleves the question: is there any bug that could be caught by unit testing that couldn't be caught just as easily by simply using the bloody software?!

Every hour you spend writing tests is an hour you could have spent fixing bugs.

Angus Glashier
Using the software is a good way to find bugs. Unit tests are a good way to ensure that these bugs have actually been fixed and stay fixed for good.
-1 every hour you spend fixing bugs in a piece of code is another hour you'll spend on the same code next week is another hour...while every hour you spend writing tests is a millisecond next week is a millisecond...
just somebody
btw, it's bugs you deployed **because** you didn't have unit tests!
just somebody
Writing tests ensures that your code is written so that it can be tested. Code written so that it can be tested must necessarily have a simpler behavior model than code that cannot be tested. Testable code is easier to understand and work with.
Aidan Cully
+1. It's not that unittests aren't useful, it's just that some things are really hard to test as units and just testing the whole app or large portions thereof as a black box gives more bang for the buck.

I think unit testing can fail when people treat it as a first class citizen, which is (seemingly to me) opposite the thinking of many TDD folk.

By this I mean, that unit tests over time have their own maintenance costs, and bring people to fear re-factoring things because they would also have to refactor tests. Being able to intelligently refactor code with the least effort possible is to me of the highest importance.

If a unit test is starting to get in the way of needed changes to the real code, or taking up too much time maintaining at the expense of project development, get rid of it. You can bring something like it back later which will probably be better thought out anyway.

This is especially the case in the transition from a project undergoing heavy development, to maintenance - this is exactly when many tests start to become more a burden than a boon. In these cases you can think of the unit tests you have built as scaffolding around a building under construction, to be discarded as better ways of working on the building are brought in and day to day challenges appear that demand new tests.

Kendall Helmstetter Gelner
Isn't the transition to maintenance, where involved developers will drift away and less familiar developers may end up working on the code exactly when you want comprehensive unit tests?
No, because then they have two bodies of code they do not understand - the actual code and the unit tests. It's better if the maintainers learn by writing the unit tests, thereby gaining an understanding of what the code is actually doing. Remember how comments can be misleading because they lie? Well unit tests can lie as well, because if you have not written them you fundamentally cannot be sure of how complete they really are (even "code coverage" cannot tell you if all possible data sets have been run through code).
Kendall Helmstetter Gelner
+2  A: 
  • Team members didn't understand the purpose of unit tests, how to write them, or how to use the tools.
  • Tests are written against concrete classes instead of mocked classes.
  • Tests are written to ensure that code calling a web service works. When the web service breaks or isn't fully functional, the test breaks.
  • Tests do unhappy things like pull every single user out of LDAP.
  • Tests take too long to run and half of them fail, so we rarely run them.
  • UI code is difficult to unit test.
  • People fall into old ways of testing, such as testing the UI manually or creating their own command-line based test client.
  • I have no way of enforcing that unit tests be written.
  • Artificial deadlines leave us no time to write unit tests.

It does not work because:

  1. How do you unit test GUI? Stuff like GUI responsiveness is hard to unit test.
  2. Incomplete unit tests, some scenarios are missed and are not tested. Although this is usually programmer oversight, not the problem with unit test practice itself.
Hao Wooi Lim

I wrote a few tests for my C++ code recently, which were just short int main() functions that called some of my functions and checked the results with assert(). I compile these tests using a Makefile and call them on make test.

Would you call that unit test, too? Should I switch to one of the test frameworks? What would the advantages be for me?

Yes I would call that unit testing and a framework would make it easier to collate results but some people advise rolling your own as you have done. If you have lots of command-line stuff, consider wrapping in Python unit tests (trivial to write) rather than a C++ framework.
Andy Dent
function == unit, you're testing individual functions -> it's unit testing
just somebody

I have had 3 major shifts in the goal posts for the project. The original scope is very fluid due to customer business model changing.

Unit tests written and re written during the first two changes were dropped in the third iteration due to time constraints and code base being largely rewritten.

I will seriously re-consider using unit tests in my next project without a VERY FIRM specification


Unit testing, which I was in favour of because it was an automated form of the regression testing I'd been using for years, failed on one significant project because nobody could tell me how to do it for an embedded, soft real-time environment. I was developing stereoscopic imaging for QuickTime and had no idea how to use unit testing or how to fake the environment.

With lots more experience, particularly working for a CTO who had a strong background in unit testing image processing applications, I would now at least write unit tests for the core algorithms by refactoring the code to be able to invoke them in isolation.

The creative step, it embarrasses me to say, that I didn't think to take was having image files saved of acceptable results and comparing with them.

It would still have required a lot of infrastructure being written, to populate data structures in the manner being filled by QuickTime and so I'm still not 100% convinced unit testing would have been suited to that project.

In summary - if you're embedded in a complex environment with rich data, mocking may be too hard.

Andy Dent
  • Lazy: it's very hard and time-killing to update all test in a big and recently changed project

  • Junior developers: no experience with separation of concerns, modularity, mocking, interfaces

  • GUI: never found an usable solution for this layer

  • Legacy code

+1  A: 

When working with a team the most difficult part with TDD is to get your other team mates to do TDD. Making a "cowboy coder" believe in TDD is daunting daunting task. They push back on TDD just because they are down right lazy & they just don't believe in writing loosely coupled code. TDD for the project crashed and burned because of the existence of lazy cowboy coders..!!

AB Kolan

Having to deal with bloated application servers and other "Enterprise" software with its claws in everything that makes it hard to write independent tests.

David Plumpton

When I started out, I bumped into isolation problems when using database access and WCF. I didn't know about mocking frameworks (like Rhino or Typemock) back then, so I didn't use one, instead I did manual mocks. This prooved time consuming and complicated so I started using mocking frameworks.

+1  A: 

The main reasons why Unit testing hasn't worked for me on the projects I have been on.

  • Management has not bought into unit testing. They would rather developers spend time on writing the application than writing tests.
  • Developers are not educated on writing unit test and what it takes to write a good unit test so that it can be run repeatedly without having to maintain it every time you run your test suite.
  • Unit test build not being run nightly or rather not integrate continuous build on the project.
  • When unit tests broke no one took the time to fix them.

These are all of my reason. I am a big believe in unit testing and I personally will not write a piece of application code without a unit test. There is a fine art to writing test so that they don't take up most of your development time.

Peter Delaney

I've been on a number of projects that employed unit testing. To a number of these projects, it added tremendous value. To others, it added little appreciable value. In no case, however, would I say the fault was with the concept of unit testing, however.

The cases in which an attempt at adding unit tests was either unsuccessful (meaning they didn't end up being written) or that they added little appreciable value were due to two primary factors: * The tests were either written after all the code had been written, or we were trying to write tests for new code, but that was based legacy code which had been originally developed without unit tests. Such code almost invariabily is written in a fashion that maes writing unit tests extremely difficult. Typically it has a lot of stuff going on with components that all heavily depend on eachother and don't expose the things necessary to write tests. Overcoming this is a huge, huge undertaking and not something that was easily supported given the time and cost constraints of these projects
* Developers did not understand proper unit testing. Often, they would write "unit tests" that either did nothing more than execute a swath of code without ANY asserts. Even after trying to help teach these people proper unit testing, I'd still some unit tests that had very little real value.

In every project in which meaningful unit tests were written from the beginning, helping define and inform the code base, they added tremendous value. I've also had limited success on adding them to existing apps that previously lacked them, but this involved a great buy in from managment for the extra time required to refactor the code to accomodate them. In these cases, we didn't try to test the old code, only to refactor as little as possible so that the new code could have tests written for it. There is an important distinction there.

+1  A: 

My guess is that unit testing often didn't work because it wasn't introduced at the beginning of a project.

  • With unit tests, it's very important to refactor your code so that it is easy to test. For example, a single class that has database access, business logic, and some display logic in it is going to be very difficult to unit test. This is easy to do when the code isn't writen yet. It's much less trivial when there's a huge chunk of existing code, and tests are being added after the fact.

  • Refactoring code without good unit test coverage is very difficult.

  • Unit tests are not only level of testing required to ensure confidence in the quality of your application. They are only the lowest level of testing; their primary value is to empower refactoring.

This is why adding unit tests to an existing body of code is often a very risky activity. If the code isn't written in a test friendly manner, refactoring it into a test friendly form is very difficult, because there are no unit tests. This chicken and egg problem can make progress very slow.

Solution: the first step is not to add unit tests.

My suggestion is that before adding unit tests to a legacy project, you should create a good functional test suite. This is easier to do, since functional tests are black-box in nature; test-friendliness of the production code is less of an issue.

Once you have good functional test coverage, you can tease separate concerns apart into domain layers (possibly rewriting a component here or there), while being confident that the application didn't break as a result of your changes. As you go, you write unit tests at the seams where you split components (or for rewritten components, TDD the whole thing).

Once this process is complete you will be left with a structure that is much easier to add unit tests to.

Characteristics of a good functional test:

Good functional tests are:

  • Black box. They don't require internal knowledge of how the app works. For example, rather than checking a database to verify that some data was recorded, consider checking another screen of the application.

  • Process oriented, in a narrative form. A good functional test doesn't look much like a good unit test. Rather than testing a specific class, you test a process. They have a longer, narrative form, with multiple steps. It's almost like explicitly testing a use case.

  • Non-functional. Good unit tests have a lot of the characteristics of functional programming. They don't have side-effects, they can be executed in any order, etc. The same is not true for functional tests. It's perfectly ok to require the steps inside of a functional test to execute in a specific order. You'll probably want a way to clean up after the entire suite is run, but individual functional tests can leave a little bit of mess behind.

  • Less specific failures. A good unit test will fail for only one reason, and one bug should cause exactly one test to fail. Failing functional tests may happen for several reasons, and one bug can cause many tests to fail; an app that can't connect to a database may fail every test.

  • Different life-cycle. You don't run functional tests TDD style as you develop. You do run your functional test suite once as you finish a feature, to demonstrate that it is finished. You also don't run them on your workstation, to avoid "works on my machine" syndrome. (Although you can run them on your workstation first to predict that they'll work in the test environment)

Sean Reilly

I'm in the "it didn't work for me because we didn't use it" (all of our apps our intranet web apps - camp. I also don't know nearly enough about it. Enough to know I want it.

I've been learning MVC and I can appreciate that part of the sell is test coverage. I pushed a little for our next project to use MVC but didn't get much in the way of positive feedback. I'm hoping to make Unit Testing a bigger focus once I'm more comfortable with MVC.


It's helped on all the projects I've used it on, but the "challenges" have been:

  • only some developers on the team wrote tests
  • tests are too brittle, and so they get commented out
  • developers don't fix tests when they break because no one makes them
  • tests take a long time so developers can't use them as actively

In my opinion, it is very sensitive on personalities + skills of programmers. You should check quality of tests and check what your team really does.

E.g. I have seen this kind of tests for whole project in team pretending development with SCRUM & TDD:

  • Tests did not have any assertions (only logs).
  • Nearly every test method contained try-catch block preventing their automatic failure in case of exception
  • Test data were presented only on development machine of concrete developer
  • Tests were not executed on regular basis
  • Etc...

This is not failure of TDD, but I wanted to present how "unit tests" could be just next way how to throw money from window.

+2  A: 

I can't say that there's ever been a time that unit testing did not work out for a project. There have been times (currently ongoing!) that portions of code will not be unit tested because the authors of the code did not care and made things hard to test. But in cases like this I just find a way around it and I cover my own code.

This always works for me. At release time I go home on time knowing my code works and the developers with the flaky code are fiddling around with something here and there until 10pm trying to fix things they didn't know were broken but QA noticed.

+1  A: 

I often have trouble making unittests work in my scientific computing code because some of the algorithms require such complex inputs that simulating an input non-trivial enough to be worth testing or figuring out by hand what the correct answer is would be nearly impossible. Usually I still unittest the lower level stuff that requires relatively simple input so that at least I know that the higher-level algorithms are built on a solid foundation, and rely on sanity checks to make sure the results of the higher level code are reasonable.

I have quite a similar problems. Creating the edge cases that happen in some combinations is hard. Usually you don't know about these edge cases until they show up.
Tomas Pajonk

Our unit tests were written by the same developers as the production code.

The production code was crap. The unit tests were worse.


Because we didn't test everything that seemed a bit too hard to test, and everything that wasn't our work ... which in the end led to something like 15% of really unit tested code...

"a bit too hard" here means code that you can't test input and output right away, because it's creating a bunch of objects behind, or accessing network resources, using timers, etc, etc

I believe the code wasn't modular enough, not really adapted to unit testing

this sounds like you're having too much responsibility within one class. Try seperating this into several classes and make start easy. I've seen it as well, you don't write tests because it is simply too hard ( you have to drag the whole world of dependencies in to make it work). That is not the intention of unit testing. Although it does point out the flaws in your design this way.
Stefan Hendriks
+1  A: 

Unit testing can't work when you are building the application from scratch, because you'll be focused on making it work, not breaking it, at first.

However, real-life situation for making unit test invaluable is refactoring. When application or part of it gets to the running condition, and users' not complaining are real proof of it, then you can say that behaviour of the system components is as it should be.

And then, you want to make some architectural changes, for various reasons, for example lifting some parts of code to meta-level, due to performance issues, due to obsolence of some library or part there of... You'll want to fix some part of existing behaviour as ''in stone'' and make unit tests for it, then refactor.

Daniel Mošmondor
I think if you approach this TDD wise, you would start with unit tests (with no code to test against). This forces you to think early about your design. With the short cycles (where you refactor code) you eventually end up with some code that works, and that is solid. I can't see why unit testing is not suitable for code from scratch, its much harder with existing code in fact.
Stefan Hendriks

Unit testing has failed every time i try to start it because

  • i can't find a book, blog, or article that explains how to apply it to my situations
  • it requires complexifying up the code base
Ian Boyd

"Also, the unit testing in this case was largely a failure because the tests were mostly aimed at aspects of the application that were never going to change, and ignored the weakest points. In this case, the weak points were the availability of network resources and the state of the data in the production environment. All of this was mocked in the test suite, and essentially useless. This incredibly heavily unit-tested application failed miserably in the field."

Was tesing of network resources really a unit test? Maybe an integration test? Maybe a system test? You do have those, don't you?

I am not saying that load & stress test should not never be at unit test level, but generally not.

This (as you describe it) was not a failure of uint test, but a failure of your overall test startegy.


It wasn't my project, but even after extensive testing the system unexpectedly failed 4 years into release because threads were used. Search for "Ptolemy" here.

joe snyder