Is there any way for executing a method multiple times, but managing connections/threads? (.NET)

Question

1. I have a method that uses a connection (e.g. a method that downloads a page).
2. I have to execute this method multiple times (e.g. download 1000 pages).
3. Doing it the synchronous and sequential way takes a long time.
4. I have limited resources ( 8 max threads and/or 50 max simultaneous connections )
5. I want to use all resources to accelerate it.
6. I know that parallelization (PLINQ, Parallel Extensions, etc.) could solve the problem, but I have already tried it, and this approach fails because of the scarce resources.
7. I don't want to reinvent the wheel that parallelizes this kind of task while managing the resources, someone must have done it before and must have provided a library/tutorial for this.

Can anyone help?

Update Things get much more complicated when you start to mix Asynchronous calls with Parallelization for maximum performance. This is implemented on several downloaders, like Firefox downloader, it gets 2 downloads simultaneously and when one of them is complete it gets the next file and so on. Maybe it seems very simple to implement, but when I implemented it I had, and still have, trouble to make it generic (useful for WebRequest and DbCommand) and to deal with problems (ie. timeouts)

Bounty Hunters The bounty will be granted to the first one that links a reliable and free($$) .NET library that provides a simple C# way to parallelize async tasks as HttpWebRequests.BegingetResponse and SqlCommand.BeginExecuteNonQuery. The parallelization must not wait for N tasks to complete to then start the next N, but it must start a new task as soon as one of the N initial ones finishes. The method must was provide timeout handling.

Answer 1

You could use the .NET System.Threading.ThreadPool class. You can set the maximum number of threads to be active at any one time using ThreadPool.SetMaxThreads().

Answer 2

Look into a counting semaphore for the connections. http://en.wikipedia.org/wiki/Semaphore_(programming)

EDIT: To answer your comment the .NET Framework has one already. http://msdn.microsoft.com/en-us/library/system.threading.semaphore.aspx

Answer 3

I would strongly recommend staying away from the threadpool except for very short tasks. If you choose to use a semaphore make sure that you only block in the code that is queuing the work items, not at the start of the workitem code or you will quickly dead lock the thread pool if your (semaphore max count * 2) is greater than max pool threads.

In practice you really can never safely acquire a lock on a pool thread, nor can you safely make calls to most async APIs(or sync APIs like HttpWebRequest.GetResponse as it also performs async ops under its covers on the thread pool).

Answer 4

1. Create a data structure to keep track of what pages have been fetched, and what still needs to be fetched. e.g. a queue

2. Using the Producer/Consumer Queue pattern, dispatch 8 consumer threads to do your fetches. That way, you know that you will never exceed your 8 thread limit.

See here for a good example.

Answer 5

Jeffrey Richter has a Power Threading Library that might help you. Its chock full of samples and is pretty powerful. I couldn't find a quick sample with connections but there are plenty of examples that might work for you in regards to coordinating multiple asynchronous operations.

It can be downloaded from here and there are several articles and samples here. Also, this link has a detailed article from Jeffrey explaining concurrent asynchronous operations.

Answer 6

Can you give more information why Parallel Linq won't work?

My point of view, your task is best suit with PLinq. If you run on 8 cores machine, PLinq will split to 8 tasks, and queue all remaining tasks for you.

Here is draft code,

PagesToDownload.AsParallel().ForAll(DownloadMethodWithLimitConnections);

I don't understand why PLinq consume up your resources. Based on my test, PLinq performance is even better than using ThreadPool.

Answer 7

See the CCR. This is the 'right' way to do it although you may find the libraries learning curve a bit to much...

Answer 8

The async WebRequest methods can appear slugggish because they block while performing DNS lookup, then switch to asynchronous behaviour. Having followed this path myself, it seems inefficient to spin up eight threads to feed requests into an API that already spins up threads to do the bulk of the work. You might reconsider some of your approaches bearing this shortcoming of the async WebRequest API. Our solution eventually involved using the synchronous API, each one on its own thread. I'd be interested in anyone commenting on the correctness of this approach.

Answer 9

Here's what I don't get: you say max 50 connections, but only 8 threads. Each connection by definition "occupies" / runs in a thread. I mean, you're not using DMA or any sort of other magic to take the load off the CPU, so each transfer needs an execution context. If you can launch 50 async requests at once, fine, great, do that -- you should be able to launch them all from the same thread, since calling an async read function takes essentially no time at all. If you e.g. have 8 cores and want to make sure an entire core is dedicated to each transfer (that would probably be dumb, but it's your code, so...), you can only run 8 transfers at once.

My suggestion is to just launch 50 async requests, inside a sync block so that they all start before you allow any of them to complete (simplifies the math). Then, use a count semaphore as suggested by Jeremy or a synchronized Queue as suggested by mbeckish to keep track of the work remaining. At the end of your async-complete callback, launch the next connection (if appropriate). That is, start 50 connections, then when one finishes, use the "completed" event handler to launch the next one, until all the work is done. This shouldn't need any kind of additional libraries or frameworks.

Answer 10

This is how you'd do it with the base class library in .net 3.5: The call to SetMinThreads is optional - see what happens with & without it.

You should handle timeouts within your replacement to DoSomethingThatsSlow

public class ThrottledParallelRunnerTest
{
    public static void Main()
    {
        //since the process is just starting up, we need to boost this
        ThreadPool.SetMinThreads(10, 10);

        IEnumerable<string> args = from i in Enumerable.Range(1, 100)
                                   select "task #" + i;
        ThrottledParallelRun(DoSomethingThatsSlow, args, 8);
    }

    public static void DoSomethingThatsSlow(string urlOrWhatever)
    {
        Console.Out.WriteLine("{1}: began {0}", urlOrWhatever, DateTime.Now.Ticks);
        Thread.Sleep(500);
        Console.Out.WriteLine("{1}: ended {0}", urlOrWhatever, DateTime.Now.Ticks);
    }

    private static void ThrottledParallelRun<T>(Action<T> action, IEnumerable<T> args, int maxThreads)
    {
        //this thing looks after the throttling
        Semaphore semaphore = new Semaphore(maxThreads, maxThreads);

        //wrap the action in a try/finally that releases the semaphore
        Action<T> releasingAction = a =>
                                        {
                                            try
                                            {
                                                action(a);
                                            }
                                            finally
                                            {
                                                semaphore.Release();
                                            }
                                        };

        //store all the IAsyncResult - will help prevent method from returning before completion
        List<IAsyncResult> results = new List<IAsyncResult>();
        foreach (T a in args)
        {
            semaphore.WaitOne();
            results.Add(releasingAction.BeginInvoke(a, null, null));
        }

        //now let's make sure everything's returned. Maybe collate exceptions here?
        foreach (IAsyncResult result in results)
        {
            releasingAction.EndInvoke(result);
        }
    }
}

Answer 11

You should take a look at F# asynchronous workflows.

You really don't want your code to be parallel but asynchronous

asynchronous refers to programs that perform some long running operations that don't necessary block a calling thread, for example accessing the network, calling web services or performing any other I/O operation in general

This is a very interesting article on this concept explained using C# iterators.

This is a great book about F# and asynchronous programming.

The learning curve is very bad (a lot of odd stuff: F# syntax, the Async<'a> type , monads, etc.) but is a VERY powerful approach and can be used in real life with great C# interop.

The main idea here is continuation: while your're wating for some I/O operations let your threads do something else!