Can't safely lock a value of a ConcurrentDictionary

Question

Hi,

I asked a question about this yesterday and got lots of helpful feedback (thanks!) but I don't think I gave enough information in the question - hence, another.

I have two threads which concurrently read two files. They put the information from these files into two ConcurrentQueues. Another two threads then come along, dequeue items from the ConcurrentQueues and put items into a single ConcurrentDictionary. When updating an item in the Dictionary, the threads may have to create a new object, or just inform a current object that more information has come in. In the latter case, sometimes a lengthy scan has to happen. Sometimes, after this scan, the object says its OK to delete it (as an attempt to save memory) and the thread removes it from the Dictionary.

My current (broken) code below:

string dictionaryKey = myMessage.someValue;

Monitor.Enter(GetDictionaryLocker);
DictionaryObject currentObject = myConcurrentDictionary.GetOrAdd(dictionaryKey, new DictionaryObject());
// we can be interrupted here
lock (currentObject)
{
    Monitor.Exit(GetDictionaryLocker);
    //KeyNotFoundException is possible on line below
    if (myConcurrentDictionary[dictonaryKey].scan(myMessage)) // Scans the message - returns true if the object says its OK to remove it from the dictionary
    {
         DictionaryObject temp;                      //   It's OK to delete it
         if (!queuedMessages.TryRemove(ric, out temp))   // Did delete work?
             throw new Exception("Was unable to delete a DictionaryObject that just reported it was ok to delete it");
    }
}

What happens is this:

As between finding the object I want in the Dictionary:

DictionaryObject currentObject = myConcurrentDictionary.GetOrAdd(dictionaryKey, new DictionaryObject());

and then locking on that object:

lock (currentObject)

, the thread can be interuppted, so theres a chance that another thread has deleted the object out of the dictionary by the time I get around to trying to access it here:

if (myConcurrentDictionary[dictonaryKey].scan(myMessage))

This then results in a KeyNotFoundException. I need some way of locking the object atomically.

As I said, I got some suggestions yesterday, but I don't understand how I could use them

• One poster mentioned that I should try to Remove items from the Dictionary first, as this is an atomic operation with ConcurrentDictionary, and then re-add them. However, I'm not sure how I would then indicate to the other thread that it should wait for the item to be re-added, rather that just thinking its a missing value and creating it.
• Another poster brought up Threading.Interlocked.CompareExchange, which I could use to mark that the object is 'in use'. But I wouldn't know how to deal with the case of an object being in use - how would I wait for it?

Some limitations I have: I have to process the ConcurrentQueues in order, so I can't give up on putting an object in the Dictionary, or come back later - I need to block. The Dictionary may contain 500,000 items or more, so I really need the O(1) lookup time of the ConcurrentDictionary.

Any ideas? Sorry about the long post

Thanks,

Frederik

Answer 1

You could change the scan line into this:

DictionaryItemType dictionaryItem;

if (myConcurrentDictionary.TryGetValue(dictonaryKey, out dictionaryItem))
{
    if (dictionaryItem.scan(myMessage))

This way you recheck whether the item still is in the dictionary and if it isn't, just not go into the scan branch.