Does lock create a pile-up of threads? / Is this a correct way to avoid a lock

Question

I have a program which receives data from unmanaged code about 500 to 700 times per second. Some of this data is usefull and needs to be processed and some of it is useless and get disgarded right away. To find out if received data is usefull I use a List of strings. My problem/question is: when I use a lock on the List to delete some or all entries, wil I get a big pileup of threads waiting to search the List?

Because deleting the entire list or parts of it is not used continuosly I now use a static Boolean. When I start with deleting I turn the Boolean to false and all data is disgarded before the list get searched. When I'm done I turn the Boolean back to true. Is this a gooed workaround or is there a better one? (I'm also asking this because testing is very time consuming at this point)

EDIT

The program is used to check if the strings in the list are correct. The unmanaged code sends data and this happens on a new thread. If the data is useful it gets displayed and the user can verify this. If the data turns out to be displayed but not usefull the user can delete the string from the List which happens on the main thread.

Answer 1

Yes - you may well get a "big pile up of threads".

I would recommend looking into a lock with reader/writer semantics as opposed to a single "brutal" exclusive lock. This should enable many readers to read your data concurrently. Only when a writer comes along to update the data will an exclusive lock be taken. Provided the number of writes is low in relation to the number of reads, you'll have very few threads "backing up".

The .Net ReaderWriterLockSlim is one possibility, but I heartily recommend you look at the OneManyResourceLock from Jeffrey Richter's Power Threading Library

Answer 2

Would it be possible to process the strings before they get added to the list and only add the required ones?

This would remove the need to delete from the list at all.

EDIT

Ok, then, instead of a list of strings create another class with two variables :

1. the string.
2. a boolean.

Instead of removing the string from the list, set the boolean to false. Then no longer search on this.

This may or may not be more efficient, it depends on the situation.

Advantage

You only need to lock each individual record, not the whole list to remove. This could save the pile up of threads.

Disadvantage

You are not clearing down your list, so you will be searching through records that would have been otherwise deleted.

You might be able to get around this by using any idle time to purge the list if needed.