views:

1295

answers:

8

Any suggestions on how to write repeatable unit tests for code that may be susceptible to deadlocks and race conditions?

Right now I'm leaning towards skipping unit tests and focusing on stress tests. The problem with that is you can run a stress test 5 times and see five different results.

EDIT: I know its probably just a dream, but if there were a way to control individual threads and cause them to execute one instruction at a time then I might get somewhere.

+3  A: 

Take a look at TypeMock Racer (it's in Beta)

edit: actually Alpha

http://www.typemock.com/Typemock_software_development_tools.html

Lou Franco
Call me sceptical. I've had race conditions I couldn't reproduce even once I figured out what was causing them. TypeMock Racer might find some but it's just a stress test. Passing doesn't mean they aren't any race conditions. Still it might be useful.
Kevin Gale
That's not really a "unit test", but it could be useful none the less.
Jonathan Allen
No, can read the IL and tell you if you have a deadlock and then put you in a situation. I just got a demo, but that's what I understood them to mean as they explained it.
Lou Franco
I agree it may be very useful but I don't believe that it's a unit test in the sense that passing means you have no deadlocks or race conditions.
Kevin Gale
I take it back, that does sound interesting.
Jonathan Allen
A: 

It is usually possible to force foreseen race-conditions and deadlocks by using things like ManualResetEvent to get each thread into the expected state before releasing it - i.e. get thread A to have the lock and wait for a signal... get thread B to request the lock, etc...

However - you might typically write such a test to investigate a suspected bug, to prove when it is fixed and that it doesn't re-surface. You would generally design around race conditions (but test them as best as is pragmatic).

Marc Gravell
Beware - waiting on an event causes a cache flush which can mask a race condition. It can be worth using for-loops for delays in this situation.
finnw
Fair enough - I was talking more about proving the *consequences* of a deadlock, and showing them fixed - but point taken.
Marc Gravell
+1  A: 

Can't think of a good automated way, but the closest I've come was to writing a unit test that would 'expose' a deadlock was by using breakpoints in addition to a unit test. I simply added some instructions on where to add the breakpoint. There is some manual effort involved, but with them you can always expose your worse case thread schedule.

Perhaps someone has figured a good way to automate this type of functionality? I could imagine automatically running the debugger, breaking one thread at a specific line, letting another one run until a specific condition, then asserting for the unit test.

runT1ME
The problem with break points is that it changes how the code behaves. Some activities like instruction reordering and inlining don't happen when the debugger is attached.
Jonathan Allen
+1  A: 

I don't think looking for race conditions really falls into the realm of unit testing. More-or-less by definition, the only way to test for race conditions is pseudo-randomly. Unless you're willing to go to the effort of formally proving the correctness of your locking strategy, you'll have to do some stress-testing.

You still need to write unit tests, to verify to correctness of the algorithms, rather than the locking strategy.

When stress-testing multi-threaded code, you'll want to test under conditions where you have one CPU per thread, where you have multiple threads sharing the CPU, and where you have more CPUs than threads (if possible).

Mark Bessey
+1  A: 

You can write a lock class that detects potential deadlocks, by looking at the ordering of the lock operations. We do this by having a thread context that all locks register with when they are acquired (can be made a DEBUG only option).

The idea is to create a graph where the nodes represents locks and a directed edge between A and B means 'lock A was being held when lock B was acquired'. Let your program run using normal loads, then check for cycles in the graph. A cycle means there is potential for deadlock even if your code didn't hit it.

Rob Walker
+1  A: 

Disclaimer: I work at Typemock.

I've seen the comments regarding Typemock Racer and I want to clarify a few things:

  1. Racer is not a "stress test tool" it uses static and dynamic analysis algorithms to discover Deadlocks in .NET code. This means that in case no Deadlock was found using Racer no deadlock exist (unless we have a bug :)).
  2. We are working hard on adding Race condition discovery to Racer - In (near) future versions we'll have Race Condition analysis but current version only finds deadlock.
  3. Finally - Racer is in it's Alpha stage which means its free to download and use by anyone.
Dror Helper
+1  A: 

Just a quick comment on (that I apparently require 50 reputation to make)

Racer is not a "stress test tool" it uses static and dynamic analysis algorithms to discover Deadlocks in .NET code. This means that in case no Deadlock was found using Racer no deadlock exist (unless we have a bug :)).

quoting from http://en.wikipedia.org/wiki/Deadlock#Detection

Detecting the possibility of a deadlock before it occurs is much more difficult and is, in fact, generally undecidable, because the halting problem can be rephrased as a deadlock scenario.

So Racer can at best do a heuristic search - unless you solved the halting problem ;)

A: 

I have previously used artificial delays in the code that are triggered by some parameters in the request. For example one request tells the server to delay write between two writes and another to do them with no delay in between.

A Mark Bessey writes, this is only useful for creating repro, not for discovering the problem.

Hemal Pandya