How to test for thread safety

Question

Do you have any advices how to test a multithreaded application?

I know, threading errors are very difficult to catch and they may occur at anytime - or not at all. Tests are difficult and the results are never for sure. Certainly it is best to carefully design and program the concurrent modules.
Nevertheless - I do not want to leave out the test aspect. So running a lot of threads that are all working on the same items can invoke threading errors, sometimes.

Any ideas or best practices to get a high hit rate of hidden threading errors?
(I am using .Net/C#)

Answer 1

Try increasing the number of threads to a large number if possible, even beyond how many will be used in a release. With lots of threads running your program, an error will appear more often since more threads are running over the code.

Double check your declarations, locks, unlocks, semaphore counts, etc and make sure they make sense.

Create a test document or spreadsheet, and using your knowledge of the code, think about where possible race conditions or deadlocks could occur.

Grab some people from the hall and do a 'hallway usability test' (Joel on Software said that I think?). Generally, people who have no idea what your program does/is about will be able to break it easily.

Answer 2

You can use some good tools to test all the threading issues like Data races, deadlocks, stalled threads etc. intel-thread-checker is one such good tool.

You can also try, CHESS by Microsoft Research

Answer 3

Good question. I usually test for race-conditions by spawning many threads and letting them wildly perform the operations which I suspect might be subject to race conditions.

Maybe you can look at PNUnit - although it's probably a little different from what you are looking for. The authors say they built it because "we needed to simulate hundreds of clients against the same server".

Answer 4

grep code for calls to threading routines. If any are found, fail the test, as your code has multi-threading bugs.

If it passes, expand your search to the parts of libraries you use, until it fails or (unlikely) is proven thread-safe (i.e. single-threaded).

Once you know you have threading bugs, the testing part of the job is done. All that remains is the small matter of finding and removing them...