What is the most difficult UnitTest case you have ever encountered?

Question

One of the most difficult test cases I have encountered is testing features that operates on files and folders. It is not easy to mock or simulate files and folders.

Answer 1

I had a event model async class that handled and FTP call and did some work on the completed event.

My first round of testing tried to create a SynchronizationContext and wait on the thread until it returned the completed event, In my case the completed event was being mocked and returned withing seconds, but without the context the tests were failing.

I ended up working around this problem by breaking it into two tests.

1. Ensure that the UploadAsync call sent a call to the FTP Service Upload method
2. Enusre that when the Completed callback updated the View.

Answer 2

Testing anything that depends on HttpContext.Current in ASP.NET. What a huge pain to mock. Fortunately, they've introduced System.Web.Abstractions and I expect that newer development will use HttpContextBase and easier to mock patterns as ASP.NET MVC does.

Answer 3

Anything involving ancient, impenetrable code, threads (no no no!) and time-sensitive stuff.

Answer 4

Threading stuff is hard to test and IO is not easy to mock. So these kind of tests are quiet hard to do.

Answer 5

Working with hardware has been a tough issue for me. It is virtually impossible to write tests against in many cases.

Answer 6

At my last job we wrote a similar unit test - it was for a class that would read files & folders relating to overlay images for MSVE. The unit test class would create a folder structure in a specific place, the unit tests would go through the folders, and then the unit test cleaned up the folders when it was done. But when this unit test ran every time a checkin occurred... it got old. :)

Answer 7

The standard answers are filesystem, network, and threading.

Personally, the trickiest case I ran into involved randomly shuffling a list. How can you assert anything about the results when the results are random and are dependent on your random number generator's internal implementation?

Answer 8

I never came up with a satisfactory answer for this one. I'd like to TDD a unit test that makes me add the 'volatile' modifier that fixes this issue in java 5+:

http://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html

I never got a satisfactory answer to this question, either:

http://stackoverflow.com/questions/537014/using-tdd-to-drive-out-thread-safe-code

So, that's another candidate.

Answer 9

Code where the results are not well-defined. For example: Imagine you want to unit-test an algorithm that recognizes handwritten text. Everybody understands what that means, but it's pretty hard to translate that into statements that can be tested by a computer. You could test how the algorithm works (e.g. it transforms the image into a stream of strokes, transform that stream into the frequency domain, apply pattern matching technique X...), but then all your tests would fail if you change the algorithm. Even if you improve it. You could test it with lots of samples, and assert that it should be able to read at least 90% of them correctly. but what if the algorithm needs some parameterization or teaching to work properly? Then your tests will again fail if you change the algorithm, because the optimal parameters are different. Also, your unit-tests might take a very long time to run.