How to write a test case for ensuring thread-safe

Question

Hi,

I wrote a thread-safe(at least the aim is that) container class in C++. I lock mutexes while accessing the member and release when finished. Now, I try to write a test case if it is really thread safe. Let's say, I have Container container and two threads Thread1 Thread2.

Container container;
Thread1()
{
    //Add N items to the container
}
Thread2()
{
    //Add N items to the container
}

In this way, it works with no problem with N=1000.

But I'm not sure this regression test is enough or not. Is there a deterministic way to test a class like that?

Thanks.

Answer 1

there is no real way to write a test to prove its safe.

you can only design it so it is safe and test that your design is implemented. best you can do is stress test it.

Answer 2

That's a reasonable starting point, though I'd make a few suggestions:

1. Run the test on a quad-core machine to improve the odds of real resource contention.
2. Instead of having a fixed number of threads, I'd suggest spawning a random number of threads with a lower bound equal to the number of processors on the test machine and an upper bound that's four times that number.
3. Consider doing occasional runs with a substantially larger number of items (say 100,000).
4. Run your tests on optimized, release (non-debug) builds.

If you're targeting Windows, you may want to consider using critical sections rather than mutexes as they're generally more performant.

Answer 3

Proving that it's safe is not possible, but for improving the stress-testing chances of finding bugs, you can modify the container's add method so looks like this:

// Assuming all this is thread safe
if ( in_use_flag == true ) {
   error!
}
in_use_flag = true;

... original add method code ....

sleep( long_time );
in-use-flag = false;

This way you can almost make sure that the two threads would try to access the container at the same time, and also check for such occurrences - thus making sure the thread-safety actually works.

PS I would also remove the mutex protection just to see it fail once.

Answer 4

I guess that you wrote a generic container and that you want to verify that two different threads cannot insert items on the same time.

If my assumptions are correct, then my proposition would be to write a custom class in wich you overload the copy constructor, inserting a sleep that could be parametrized.

To test your container, create an instance of it for your custom class and then in the first thread, insert an instance of the custom class with a long sleep, meanwhile you start the second thread trying to insert an instance of the custom class with a short sleep. If the second insertion comes back before the first one, you know that the test failed.