Multiple Threads processing a Lists data

Question

Hi all,

I have a list of 300,000 + items. What i am currently doing with the list is validating the Address and writing both the original address and corrected address to a specified file. What i'd like to do is split the list evenly among a given number of threads and do processes on them concurrently. Can anyone help me with an example on how i can go about doing something like this?

Thanks

Answer 1

Conceptually, it's fairly simple, given a couple of assumptions:

• You're not changing the list during processing.
• You know the number of threads ahead of time.

Basically, your algorithm is this:

1. Split the list into mostly-even sections based on the number of threads.
2. Give each thread the start and end index of its section, and the output file.
3. In each thread:
   a. Process an item
   b. Lock access to the output file
   c. Write the original and corrected address
   d. Unlock access to the output file
   

Answer 2

If you're working in 2.0 and the list is only being used in a read only fashion (not being changed while this processing is occuring) then you can simply divide up the indexes. For instance ...

public void Process(List<Item> list, int threadCount) {
  int perThread = list.Count < threadCount ? list.Count : list.Count / threadCount;
  int index = 0;
  while ( index < list.Count ) {
     int start = index;
     int count = Math.Min(perThread,list.Count-start);
     WaitCallBack del = delegate(object state) { ProcessCore(list, start, count); };
     ThreadPool.QueueUserWorkItem(del);
     index += count;
  }
}

private void ProcessCore(List<Item> list, int startIndex, int count) {
  // Do work here
}