Queue<T>.Dequeue returns null

Question

I have a scenario where

• multiple threads are pushing data on a Queue

• only ONE thread is processing data using code below

code -

  while ( Continue )
  {
        while ( queue.Count > 0 )
        {
             MyObj o = queue.Dequeue();
             someProcess(o);
        }
        myAutoResetEvent.WaitOne();
  }

But sometimes, queue.Dequeue() returns null in the scenario above What gives ?

Answer 1

Make sure nothing's pushing null values into the queue. nulls are allowed as queued values. Also, according to this document, only Queue<T>'s static members are thread-safe, so beware of doing reading and writing across threads.

Answer 2

You need to synchronise the access to the queue. Put lock statements around all code sections that access the queue (both reading and writing). If you access the queue simultaneously from multiple threads the internal structures may be corrupted and just about anything can happen.

Answer 3

You need to read this blog post.

Also, here's a very minimal skeleton of a "channel" for communication between threads:

public class Channel<T>
{
    private readonly Queue<T> _queue = new Queue<T>();

    public void Enqueue(T item)
    {
        lock (_queue)
        {
            _queue.Enqueue(item);
            if (_queue.Count == 1)
                Monitor.PulseAll(_queue);
        }
    }

    public T Dequeue()
    {
        lock (_queue)
        {
            while (_queue.Count == 0)
                Monitor.Wait(_queue);

            return _queue.Dequeue();
        }
    }
}

Answer 4

Why don't you solve the problem that way?

while (IsRunning)
{
    do
    {
        MyObj myObj = queue.Dequeue();
        if (myObj != null)
        {
            DoSomethingWith(myObj);
        }
    } while (myObj != null);

    myAutoResetEvent.WaitOne();
}

Update

Ok, after reading Earwickers comment and all the other answers, you are all right and i'm just false. So please don't use the code above in multithreaded contexts .

Answer 5

You say:

multiple threads are pushing data on a Queue

The Queue<T>.Enqueue method is not thread-safe. This means that work gets done within the Enqueue method that needs to be synchronized if multiple threads are calling it. A simple example would be updating the Count property. It's a safe bet that somewhere in the Enqueue method there's a line that looks something like this:

++count;

But as we all know, this isn't an atomic operation. It's really more like this (in terms of what's actually happening):

int newCount = count + 1;
count = newCount;

So say the count is currently 5, and Thread 1 gets past int newCount = count + 1... then Thread 1 thinks, "OK, the count is now 5, so I'll make it 6." But the very next operation that gets executed is where Thread 2 gets to int newCount = count + 1 and thinks the same thing as Thread 1 ("the count is now 6"). So two items have just been added to the queue, but the count only went from 5 to 6.

This is just a very basic example of how a non thread-safe method like Queue<T>.Enqueue can get messed up when access is not synchronized. It doesn't specifically explain what's happening in your question; my intention is simply to point out that what you're doing is not thread-safe and will cause unexpected behavior.

Answer 6

Guffa is correct, having multiple threads reading and writing to the queue will cause problems, because Queue<T> is not thread safe.

If you're on .NET 4, use the ConcurrentQueue<T> class, which is thread safe. If you're not on .NET 3 or earlier, you can either do your own locking, as Guffa pointed out, or use a 3rd party library.

Answer 7

If you happen to be using non-generic Queue (not that I advise to use it), you can use Queue.Synchronized method to get a thread-safe wrapper:

Queue queue = Queue.Synchronized(new Queue());

Otherwise you should take care of locking yourself, as others suggest.