What might cause a Stack Overflow during linq iteration of Dictionary?

Question

I have the following dictionary:

Dictionary<long, ChangeLogProcess> _changeLogProcesses = 
    new Dictionary<long, ChangeLogProcess>();

I have a method that attempts to get the next changelogprocess in the dictionary of a particular status (If there are no items of a particular status it returns null):

 var changeLogProcesses =
     from entry in _changeLogProcesses
     where (entry.Value.Status == status)
     select entry.Value;

 changeLogProcess = changeLogProcesses.FirstOrDefault<ChangeLogProcess>();

However, during execution it is throwing a stack overflow exception during the linq query? I have done numerous tests to make sure that there are items in teh list and so on but the problem persists?

It's worth noting that this method is part of a service that is running in a multi threaded environment. The linq query above (and all access to it, such as items added/removed to the list, or status changes to the items in the list) are all wrapped in ReaderWriterLockSlim write locks. Again, I have debugged it extensively to make sure there is never anymore than a single thread accessing the list at any time.

What might cause it to stack overflow, as apposed to some possible other errors such as a modification of the list during the query? (again I'm there is only a single thread accessing the list at any one time)

EDIT: as requested the getter and setter code:

 public ChangeLogProcessStatus Status 
 {
    get { return _status; }
    set
    {
        //more that one place can initiate a retry now, so retry count is handled in the property setter
        if (PreviousStatus <= ChangeLogProcessStatus.Waiting && value >= ChangeLogProcessStatus.RetryWaiting)
        {
            this.ChangeLog.Tries++;

            //If it's retry waiting, remove this last service machine from the 
            //list so it can try it again because it's not an error
            if (value == ChangeLogProcessStatus.RetryWaiting && _previousServiceMachineIds.Count > 0)
            {   
                _previousServiceMachineIds.RemoveAt(_previousServiceMachineIds.Count() - 1);
            }
        }

        PreviousStatus = _status;
        _status = value;

    }
}      

LAST EDIT - I've removed the previous examples as the problem did not exist in that code.

It turns out it was in a different part of the application, and it was a very hard to find piece of recursion. It was a coincidence that the stack overflow error was raised duringthe linq query, which as a result was being called 420000+ times recursively.

All answers below were helpfull and on the right path to finding the problem in multi-threaded apps, however the first answer definitly emphasized recursion as the problem which is what it turned out to be (although it wasn't one of the property accessors as seemed obvious).

THanks again

Thanks

Answer 1

Check the property on the ChangeLogProcess class to make sure that it isn't self-referential. This is, I think, the most likely cause of a stack overflow exception in this case.

Ex:

 private ChangeLogStatus status;
 public ChangeLogStatus Status
 {
      get { return this.Status; }  // instead of this.status
      set { this.status = value }
 }

Another possible alternative is in the equality check for status. Have you overridden Equals() for ChangeLogStatus? Check there to make sure you don't have any self-referential code (or at least a way of terminating the recursion).

Answer 2

I've noticed that some collections behave very badly when touched by two threads at the same time.

You are actually allowing more than one thread to touch the collection at the same time. RWLS allows multiple threads to access during read operations and lock on a write operation. So two threads could be reading, i.e. touching, the collection at the same time.

My suggestion would be to change the RWLS to a simple lock() and try to repro the stack overflow.

If this fixes your issue, I'd suggest thinking about moving to 4.0 to take advantage of the concurrent collections. Alternatively, you might want to construct your own thread-safe collection cough reflector cough so you can control this situation better.

Answer 3

http://msdn.microsoft.com/en-us/library/xfhwa508.aspx

Thread Safety

Public static (Shared in Visual Basic) members of this type are thread safe. Any instance members are not guaranteed to be thread safe.

A Dictionary<(Of <(TKey, TValue>)>) can support multiple readers concurrently, as long as the collection is not modified. Even so, enumerating through a collection is intrinsically not a thread-safe procedure. In the rare case where an enumeration contends with write accesses, the collection must be locked during the entire enumeration. To allow the collection to be accessed by multiple threads for reading and writing, you must implement your own synchronization.

Emphasis mine.