Weird problem when using ThreadPools to search through object lists

Question

I have these container objects (let's call them Container) in a list. Each of these Container objects in turn has a DataItem (or a derivate) in a list. In a typical scenario a user will have 15-20 Container objects with 1000-5000 DataItems each. Then there are some DataMatcher objects that can be used for different types of searches. These work mostly fine (since I have several hundered unit tests on them), but in order to make my WPF application feel snappy and responsive, I decided that I should use the ThreadPool for this task. Thus I have a DataItemCommandRunner which runs on a Container object, and basically performs each delegate in a list it takes as a parameter on each DataItem in turn; I use the ThreadPool to queue up one thread for each Container, so that the search in theory should be as efficient as possible on multicore computers etc.

This is basically done in a DataItemUpdater class that looks something like this:

public class DataItemUpdater
{
    private Container ch;
    private IEnumerable<DataItemCommand> cmds;

    public DataItemUpdater(Container container, IEnumerable<DataItemCommand> commandList)
    {
        ch = container;
        cmds = commandList;
    }

    public void RunCommandsOnContainer(object useless)
    {
        Thread.CurrentThread.Priority = ThreadPriority.AboveNormal;
        foreach (DataItem di in ch.ItemList)
        {
            foreach (var cmd in cmds)
            {
                cmd(sh);
            }
        }
        //Console.WriteLine("Done running for {0}", ch.DisplayName);
    }
}

(The useless object parameter for RunCommandsOnContainer is because I am experimenting with this with and without using threads, and one of them requires some parameter. Also, setting the priority to AboveNormal is just an experiment as well.)

This works fine for all but one scenario - when I use the AllWordsMatcher object type that will look for DataItem objects containing all words being searched for (as opposed to any words, exact phrase or regular expression for instance).

This is a pretty simple somestring.Contains(eachWord) based object, backed by unit tests. But herein lies some hairy strangeness.

When the RunCommandsOnContainer runs using ThreadPool threads, it will return insane results. Say I have a string like this:

var someString = "123123123 - just some numbers";

And I run this:

var res = someString.Contains("data");

When it runs, this will actually return true quite a lot - I have debugging information that shows it returning true for empty strings and other strings that simply do not contain the data. Also, it will some times return false even when the string actually contains the data being looked for.

The kicker in all this? Why do I suspect the ThreadPool and not my own code?

When I run the RunCommandsOnContainer() command for each Container in my main thread (i.e. locking the UI and everything), it works 100% correctly - every time! It never finds anything it shouldn't, and it never skips anything it should have found.

However, as soon as I use the ThreadPool, it starts finding a LOT of items it shouldn't, while some times not finding items it should.

I realize this is a complex problem (it is painful trying to debug, that's for sure!), but any insight into why and how to fix this would be greatly appreciated!

Thanks!

Rune

Answer 1

It's a bit hard to see from the fragment you're posting, but judging by the symptoms I would look at the AllWordsMatcher (look for static state). If AllWordsMatcher is stateful you should also check that you're creating a new instance for each thread.

More generally I'd look at all the instances involved in the matching/searching process, specifically at the working objects being used when multithreaded. From past experience, the problem usually lies there. (It's easy to look too much at the object graph representing your business data Container/DataItem in this case)