backgroundworkers or threadpools

Question

I am trying to create an app that allows multiple search requests to occur whilst maintaining use of the user interface to allow interaction. For the multiple search requests, I initially only had one search request running with user interaction still capable by using a backgroundworker to do this. Now I need to extend these functions by allowing more search functions and basically just queueing them up. I am not sure whether to use multiple backgroundworkers or use the threadpool since I want to be able to know the progress of each search job at any time.

If I was using the threadpool all I would do is add this in the loop which gets called each time a search request is made

 ThreadPool.QueueUserWorkItem(AddressOf Search)

but if use backgroundworkers this is the only way I know how and I am not how to add these to anything but perhaps an arraylist and I can call reportprogress from each bgw.

edit:

so for example this is my current code

 For Each Thread In ThreadList
                    'Thread.Sleep(500)
                    SyncLock Me

                        If searchChoice = "google" Or fromUrl.Contains("google") Then
                            links = parsingUtilities.GetGoogleLinksFromHtml(fromUrl, html, searchItem)
                            posts = parsingUtilities.GetPostLinksFromHtml(links)
                            If links.Count = 0 Then
                                Exit Sub
                            End If
                            Exit For
           .....

so in the above code links and posts are arraylists I use to get the urls i need and they are used for different searchchoices and I initially had the synclock on the links and posts but someone else told me to use the synclock me instead. So from your point I should I assign a separate data control for each search control and after sufficient time lock the appropriate one and transfer it to write it. thanks

Answer 1

I generally leave the threadpool alone. It's a process wide resource that can be configured to have different sizes, especially in webapps. However, since your application sounds like a client side form application, you won't be sharing a threadpool with any other apps and you can configure it to your needs.

Your use case fits the threadpool better than most but there's not a huge advantage to doing so.

Answer 2

You can still get progress updates with BackgroundWorkers, so I don't see a reason to stop using them.

BackgroundWorker itself uses the thread pool to recycle threads AFAIK. So you could probably still limit the size of the pool and keep using BWs.

Answer 3

You can use the ThreadPool in the same way you use BackgroundWorker.

The only difference is that, with the ThreadPool, you'll need to use Dispatcher.Invoke or Control.Invoke to marshal your progress and completion events back onto the UI thread yourself. However, the ThreadPool lets you easily queue up and run as many tasks as you wish.

Answer 4

I think that you should use the thread pool, because from what you've written you will have even more threads being spawned by the background threads.

In designs like this I have seen the system overloaded by thousands of threads. A thread pool is easier to manage because you have one place to set limits on threads. Some jobs may need to wait before they can get their work done but this is better than overloading the entire system.

Update:
I didn't know this, but it seems BackgroundWorker uses the ThreadPool so you are not in danger of an exploding number of threads. The system I saw exploding with thousands of threads was written in C++.