ThreadPool in .NET is queue based worker pool, however its used internally by ASP.NET host process, so if you try to utilize ThreadPool more, you may reduce performance of Web Server.
So you must create your own thread, mark it as background and let it poll every few seconds for job availability.
The best way to do is, create a Job Table in database as follow,
Table: JobQueue
JobID (bigint, auto number)
JobType (sendemail,calcstats)
JobParams (text)
IsRunning (true/false)
IsOver (true/false)
LastError (text)
JobThread class could be like following.
class JobThread{
static Thread bgThread = null;
static AutoResetEvent arWait = new AutoResetEvent(false);
public static void ProcessQueue(Job job)
{
// insert job in database
job.InsertInDB();
// start queue if its not created or if its in wait
if(bgThread==null){
bgThread = new Thread(new ..(WorkerProcess));
bgThread.IsBackground = true;
bgThread.Start();
}
else{
arWait.Set();
}
}
private static void WorkerProcess(object state){
while(true){
Job job = GetAvailableJob(
IsProcessing = false and IsOver = flase);
if(job == null){
arWait.WaitOne(10*1000);// wait ten seconds.
// to increase performance
// increase wait time
continue;
}
job.IsRunning = true;
job.UpdateDB();
try{
//
//depending upon job type do something...
}
catch(Exception ex){
job.LastError = ex.ToString(); // important step
// this will update your error in JobTable
// for later investigation
job.UpdateDB();
}
job.IsRunning = false;
job.IsOver = true;
job.UpdateDB();
}
}
}
Note
This implementation is not recommended for high memory usage tasks, ASP.NET will give lots of memory unavailability errors for big tasks, like for example, we had lot of image uploads and we needed to create thumbnails and process them using Bitmap objects, ASP.NET just wont allow you to use more memory so we had to create windows service of same type.
By creating Windows service you can create same thread queue and utilize more memory easily, and to make communication between ASP.NET and Windows Service you can use WCF or Mutex objects.
MSMQ
MSMQ is also great, but it increases configuration tasks and it becomes difficult to trace errors sometimes. We avoid MSMQ because lot of time we spend looking for an answer of problem in our code where else MSMQ configuration is problem and the errors sometimes dont give enough information of where exactly is the problem. In our custom solution we can create full debugger version with logs to trace errors. And thats biggest advantage of Managed Programs, in earlier Win32 apps, the errors were really difficult to trace.