How to solve this specific threading problem

Question

Using the code from the following article, I implemented an own ThreadPool: http://www.developer.com/net/article.php/3783756

This is what I want to achieve: Triggered through a Timer, a service should query a database every 5 seconds for new jobs to execute. A Job is basically only the information about a commandline program that needs to be run with arguments.

Up to 50 or more of these programs should be able to be executed at the same time. A program can be running a couple of seconds, minutes and also hours. The service needs to be in control of these programs at all times, i.e. it must be able to kill a program on request for instance.

Using the ThreadPool implementation from above, I startet to queue the programs to be executed and could see when the service indeed executed them. No problem so far. However, the mechanism here works like this:

The ThreadPool creates a workerthread and starts it. Whenever a program is queued, the workerthread notices this and calls a delegate that essentially instantiates a System.Diagnostics.Process object and starts the external program. The thread then is finished with its work and would be able to start further programs. However... when theres no program to start, an idle timer makes the threadmanager kill the thread and thus interrupt the Process that has been started.

This is not what I need. Does anyone here have an idea, how to handle the scenario I described better?

Answer 1

1) Why is the death of a thread in this process resulting in the death of the other, started process? If you answer that question, you'll have solved your problem.

2) That looks like a pretty lousy, and fairly naive, ThreadPool article. Check out Joe Duffy's series on a custom thread pool (part 1, part 2, and part 3). Code such as that is surprisingly intricate, and a significant maintenance burden on its own.

3) (The real answer) Why are you using a threadpool to do this at all? Your threadpool only ever uses one thread at a time, why not just use a timer and trigger your main thread? Does your app do other stuff besides in a UI that you need to keep responsive?

Get rid of the threadpool, there's no need for it, it's making your life difficult, and threadpools in general are not designed to host long-running tasks. That's what individual threads are for. If you must trigger the timer on a separate thread, simply create a Thread to handle it and use this one, identical thread to spawn all your processes. You can then track your process state in one, central, sensible location. :)

Answer 2

Are you sure that a thread pool is the optimal way of handling this? Spawning a new thread for each process which will be mostly idle but has to be present until the process terminates seems like a waste of a thread to me.

I would implement all this with a single thread and a dictionary of processes. The thread would periodically query the database and all the processes to see what actions need to be done.

Answer 3

AFAIK, processes spawned by Process.Start will continue to run even if the thread calling Process.Start exits. The following code illustrates this. LongRunningApp.exe will continue to run after the main program exits:

static void Main(string[] args)
{
    Process p = new Process();
    ProcessStartInfo psi = new ProcessStartInfo(@"C:\LongRunningApp.exe");
    psi.CreateNoWindow = true;
    psi.UseShellExecute = false;
    p.StartInfo = psi;
    p.Start();
    Console.ReadLine();
}