Accessing array from multiple threads

Question

Let's say I have two arrays:

int[] array1 = new int[2000000];
int[] array2 = new int[2000000];

I stick some values into the arrays and then want to add the contents of array2 to array1 like so:

for(int i = 0; i < 2000000; ++i) array1[i] += array2[i];

Now, let's say I want to make processing faster on a multi-processor machine, so instead of just doing a loop like above, I create two threads. One of which I have process the first 1000000 elements in the array, the other I have process the last 1000000 elements in the array. My main thread waits for those two threads to notify it that they are finished, and then proceeds to use the values from array1 for all kinds of important stuff. (Note that the two worker threads may not be terminated and may be reused, but the main thread will not resume until they have both notified it to do so.)

So, my question is: How can I be sure that the main thread will see the modifications that the two worker threads made to the array? Can I count on this to happen or do I need to go through some special procedure to make sure the worker threads flush their writes to the array and the main thread discards its cached array values?

Answer 1

You will probably be fine if you are using callbacks when they are done modifying the arrays, but if there is any question using a Lock will ensure the other threads have let go of the arrays.

lock (array1)
{
}

http://msdn.microsoft.com/en-us/library/c5kehkcz(VS.71).aspx

Answer 2

You need a memory barrier to ensure that the worker thread's writes to the array are visible to the main thread in the order you expect.

Whether or not you need an explicit memory barrier depends on how you notify the main thread. Waiting on most synchronization primitives, such as events, provide an implicit barrier, so no change would be required on your part. Polling a global variable does not provide a barrier.

If you need an explicit barrier, use Thread.MemoryBarrier.

Answer 3

As long as you didn't make a copy of the array in the main thread, I don't think you should have to do anything. Just wait until the worker threads are finished.

Answer 4

How can I be sure that the main thread will see the modifications that the two worker threads made to the array? Can I count on this to happen or do I need to go through some special procedure to make sure the worker threads flush their writes to the array and the main thread discards its cached array values?

You don't need any special handling here - you'll always be working off the same objects in memory.

Also, since each thread will work on a separate portion of the array, no locking is necessary.

However, if you're only doing a simple addition, the overhead of threading and synchronizing back to the main thread ~might~ outweigh the benefits gained... If you do this, profile to make sure that it's providing a net-gain.

Answer 5

If you break up the index range into non-overlapping ranges as you suggested, then provided the array is created in shared memory (i.e. not by each thread), then no locks are required.

Answer 6

If you're lucky and have the option to use .NET 4.0 then you can just write:

Parallel.For(0, 2000000, i => { array1[i] += array2[i]; });

You don't need any explicit locking or synchronization, because:

• Each task (execution of the for loops body) affects disjoint part of the array
• Parallel.For waits until all tasks finish before it returns, so there will be an implicit memory barrier.