Read/Write Locks

Question

As part of a project at work, I implemented a Read/Write lock class in C++. Before pushing my code to production, what sort of tests should I run on my class to be sure it will function correctly.

I've obviously performed some sanity tests on my class (making sure only one writer can access at a time, making sure releases and claims increment and decrement properly, etc.)

I'm looking for tests that will guarantee the stability of my class and to prevent edge cases. It seems testing multi-threaded code is much harder than standard code.

Answer 1

since you've implemented a read/write lock then obviously you should test it in multi-threading environment. Test scenarios such as multiple reading threads should not be blocked when there are no write operations and running thousands of read/write operations over several hours should not cause deadlock might be a good start.

Answer 2

It is very difficult to test multi-threaded code, so you should supplement your tests with a detailed code review by colleagues experienced in writing multi-threaded applications.

Answer 3

Run two threads that both do this

  long long int j = 0;
  long long int oldJ = 0;
  while (true)
  {
      for (long long int i = 0; i <= 100000; ++i) 
       // try and make it 64 bits to be sure its non-atomics :)
      {       
          oldJ = j;      
          YourRead(j) // read in j
          assert(j == (oldJ + 1));
          SleepSomeRandomPeriod(  );
          YourWrite(i)
          SleepSomeRandomPeriod(  );
      }
  }

Answer 4

I guess that you can start by looking at tests included in well-established code. For example the pthreads implementation of GNU libc (nptl) includes read-write locks and some tests.

$ ls nptl/tst-rwlock*
nptl/tst-rwlock1.c
nptl/tst-rwlock10.c
nptl/tst-rwlock11.c
nptl/tst-rwlock12.c
nptl/tst-rwlock13.c
nptl/tst-rwlock14.c
nptl/tst-rwlock2.c
nptl/tst-rwlock3.c
nptl/tst-rwlock4.c
nptl/tst-rwlock5.c
nptl/tst-rwlock6.c
nptl/tst-rwlock7.c
nptl/tst-rwlock8.c
nptl/tst-rwlock9.c

Answer 5

Use ASSERT/assert often to test all of your assumptions and to check pre and post conditions.

Answer 6

If you are able to use Boost in your work code at all, you should use the shared_mutex class, which implements read/write locking.

Even if it doesn't 100% suit your needs, you should use the ideas in the code for your code, and, if the Boost code has tests for the shared_mutex (I haven't checked), you should add them to the tests you have.

Answer 7

IBM have an analysis tool for detecting threading issues in Java. Perhaps there is something similar for C++?

Answer 8

Make sure you try your stress test on a machine that truly has multiple CPU's. That will usually uncover more multithreaded problems than anything run on a single CPU machine.

Then test it on machines that are 64-bit, faster CPUs, more CPUs, etc.

And as @onebyone.livejournal.com says, use a machine with non-coherent memory caches; although, according to the NUMA article on Wikipedia, that may be difficult to find.

Certainly using the code on as many different machines as possible can't hurt, and is also a good way to uncover issues.

Answer 9

Generally I would offer guidance to avoid implementing your own locks, unless you have proven than an existing and stable implementation doesn't meet your performance needs.

Testing and building synchronization primitives can be tricky and non intuitive.

The guidance to use boost::shared_mutex is quite wise, if you're on the Win32 platform I would guide you to use the Slim Reader Writer Locks if possible because they are robust and fast.

Though it won't help for a production product today, in Visual Studio 2010 Beta 1 we've adding a reader_writer class which while not cross platform will be part of the VS2010 redist.

Answer 10

Make sure you test on a machine with multiple CPU cores or at least a CPU with hyperthreading. There are many problems with mulitiple threads that only occur, or that occur much more frequently when threads are really running on parallel on different CPUs.